•DOCUMENT RESUME- 



ED 248 398 



CE 039 7.31 r 



AUTHOR 
TITLB 

INSTITUTION 

SPONS AGENCY 

PUB DATE 
CONTRACT 
NOTE 

P.UB 'TYPE 

EDRS PRICE 
•DESCRIPTORS 



IDENTIFIERS 



DUsewicz, Russell A. 

Assessment of the Impact of the Bureau of Vocational 
Education's Handicapped Program. Final Report. 
Research for Better Schools, Inc., Philadelphia, 
Pa. " , . , 

Pennsylvania SUate Dept. of Educeition, Harrisburg. 
Bureau of Vocational and Technical Education. 
3^ Jun 84 ' ■ • 

8r-40ic ■ . 

146p.; For a related, document , see CE 039 732. 
Reports - Evaluative/Feasibility (i42) 

MF01/PC06 Plus Postage. 

Ad.ult Vocational Education; "Agency Cooperation; 
*Disabilities; Educational Research; Job Skills; Job' 
Training; *Outcomes 6£ Education; Postsecondary 
Educatipn; Program "Descriptions; *Program 
Effectiveness; Program Evaluation; Secondary ^• 
Education;^ State Aid; State Departments of Education; 
*State Programs; . *Vocational Education 
*Impact; Impact Studies; *Pennsylvania 



ABSTRACT ' > 

Objectives of this assessment of Pennsylvania's • 
vocational programs for the handicapped were to describe and evaluate 
the programs' impact and explore the local capability for offering 
vocatioiT^al /program services' to the handicapped. Study methodology was 
guided by 23 evaluation questions representing five major study 
c>5m]^onents : problems and needs, priorities; programs and activities, 
outcomes, arid policies. Data sourCes were background -information and- 
existing documentation and files, inteijviews with Pennsylvania 
Department of Education officials, interviews with local education 
agencies, and a mail, .survey of projects. Pennsylvania's approach to 
fundi'ng vocational education programs for the handicapped was • 
principally fiscal; state funding .was distributed through an 
allocation method. The overwhelming riJS'jority of the programs' were at 
the secondary level and represented continuation programs. The most 
frequent program acti\rities were specific job skill training, 
individualized prograriu^ng, general motivationai||;/skills training," and 
pre-employment services, ^ly about half of the activities 
represented "best practice." Little state-level interagency 
cooperation existed. Generally, the needs of progran^^ participants ^ 
were met* Recommendations we^e made in these areas: definition of' 
target population, method of funding allocation, exemplary project 
identification and dissemination, use^of "best practice" techniques, 
enhancement of local program evaluations/ interagency cooperation,' - 
and a ^stronger state education department leadership role. ' 
instruments are appended^) (YLB) ^ ' \ 
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- ASSESSMENT OF THE IMPACT OF THE 
BUREAU OF VOCATIONAL EDUCATION'S 
HANDICAPPED PROGRAM ^ . 

-. \ 

. EXECUTIVE SUMtdARY . \ 



■ * Objectives ^ 

The purpose of this study is grounded in the. Vocational Education Act 
of 1976 (P.W 94-482). While this legislation made no -major changes to ^he 
basic goals o^f vocational educatio'n^ it substantially changed the 
mechanisms 'employed to further and enforce these goals.' Particularly 
noteworthy in thfs regard are two specific areas where states have been 
given increased responsibilities for delivery of quality vocational 
education: states are required to provide a full' range of vocational 
educational opportunities for the hkndlcapped, and states are required to 
evaluate all vocational education programs at least eveiy five years. This 
latter requirement is especially relevant to the pr;esent study and 
represents the mandate for local program evaluation -conducted at the state 
level. RBS was selected to conduct such an evaluation of Pennsylvania s 
vocational programs for the handicapped, specifically covering the years 
from 1979-80 to 1981-82 .inclusive. The evaluation was condticted during the 
1983-84 year and was completed in June of 1984. The study objectives were 
to describe the programs, evaluate their impact, and explore the local 
capability for offering vocational program se.rvices to the handicapped. 
' These objectives were aimed at satisfying tl)e overall goal of providing 
usef.4l evaluative iiiformation to the SEA for policymaking purposes'. 
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•?• • * Persjpectlves 

Even with the recene interest, support, aijd programmatic effort, 

researchers have indicated that vocational education for the handicapped Is 

I ' 

still invits early stages of development and several needed areas of 

* 

improvement have been identified. 'A needs assessment by 'the Natiortal 
Assoc iatio/ of State Boards of Education (1979) identified several 
deficiencies, including: interagency copperation, personnel preparation, ' 

f und ing me thodology ) se rv ice de livery and, .prx)gi:aw.,£>f>.tiftnaj.... Rr ogy^am „ 

evaluation, and services to minority handicapped youth. Studies by 
researchers at the -University of Illinols/and Pennsylvania State University 
recommended impravements regarding involving parents in the vocational 
education process, Expanding and refining administrative regulations, 
involving the business and industry sector, identifying exemplary 
practiceo, expanding teacher education activitieV^, promoting state^ 
leadership skilly, and coordinating service delivery systems (i^helps & 
Thornton, 1979)/ in light of these needs, it is alarming that^ funding for 
. federal -programs which focus on special need populations is declining. 
Under these conditions, sjJ'stematic evaluations focusing on policy issues" 
becom^ very important, indeed. The present study represents such a 
.systematic evaluation. . ' * * • 

Methods 
' — 1 ^ 

/ 

The methodology of the study was guided by a set of twenty-three 
evaluation questions whiph were based on SEA j^nterests. These evaluation 
questions represented five major study copponent^ which served as a , 
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framework for the study in general, and for the analysis and reporting oi; 
problems and needs, priori tiW, «fctiviti6s and programs,^ outcome^, and 

i ' ■ ' . 

policies ♦ - ^ ' ^ 

Data Sources 
♦ 

Five data sources were utilized in the study: background , information 

» 

and existing documentation and files> SEA interviews, LEA interviews, and a 
mail survey. Extensive background inforiiation was collected including, 
state plans, jiports > evaluatYphs ^ and proj ect monitoring summ^^ HTeT 



were collected for a total of 334 vocational education projects for the 

handicapped and were placed in ^^computerized management information 

system. Four SEA staff involved in the vocational program for the 

handicapped were interviewed* The LEA interviews consisted, of two types ^' \ 

s * 

telephone interviews -and on-site personal interviews^ Thirty-nine local 

project staff were interviewed — 27 via telephone and 12 via* site visits 

t 

and personal interviews. — out of a total sample of 93 projects. The ^ 
balance of the projects, in the sajnple (54) were surveyed' by means of a mail 
questionnaire. The^total sample of 93 projects was roughly representative 

of all projectf types but with the greater proportion consisting *of more ^ 

■ \ . ■ 

h ■ 

recent projects* An analysis plan related. each .of . thel evaluation questions 
to one or more of th# data sources^ Analyses- were primarily of the 
descriptive statistical type* 

Results 

♦ 

It was found that Pennsylvania's apjpror»ch to funding vocational 
education programs for the handicapped is principally fiscal' rather than 
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programmatic . j5tate priorities are more reactive thaiTp^active in that 
they are based upon announced federal policies, with few' changes. State 
guidelines for funding of vocational ecTucation programs for the handicapped 
appear to be fairly clear and generally meet the information needs of local 
education agencies. State funding is distributed through an allocation 
method rather than a competitive process. This has resulted in the • 
distribution of over li million dollars in funaing across 334 projects over 
the 3 year period under study. " ^ 

' In terras ©f activities and programs, the overwhelming majority • of 



'the programs funded by the 'state (over 80 percent*) are at the-secondary 
level and represent continuation progran*s 'funded year after year. Prituary 
goals of sucfi, projects tend to^be: acquisition of vocational 

> r _ • 

skills/competencies i raain^treaming into regular vocational classes, 

development of pre-employment skills, and p-lacement' in full/part-time jobs. 

i 

The most frequent activities of such programs^ are: specific job skill 
training, individualizea programming, 'general motivational skills training, 
and pre-employment services. Only about half of the activities vsithin 
funded projects represent what might be considered "best practice". In the 
field. ' ' 

- . The projects funded appear to serve individuals^ handicapping categories 
to the following extent: ,25% learning disabled, 10% severely eractionally 
disturbed, 59% mentally retarded, and 6% physically handicapped. Little in 
the way of interagency cooperation exists presently at the state level with 
regard to the funding and administratlAn of these projects. 



Generally, the needs of the handicapped program^participants wilre 

found to be met, with the majority achieving positive outcomes and about 

50% of those graduating securing employmeii|j:* It was not possible to - 

det.ermine what':^rograins worked b^st for whom because of a general lack of 

objective evaluative evidence concerning project impact ♦ 

.Less, th^n half (about ^0 percent) of the projects funded to provide 

vocationaJ^ education 'to the handicapped indicated that they "definitely" or 

"possibly" >^ec>uld continue their programs without further ^tate funding* 

Issues and activities requiring fuxtheT- atientioa^s -indicated -b^^^ 

■ \ . 

study seem to be strongest in th'e following area^: programmatic technical 

assistance, allocation of federal/stat6 funds, exemplary program identifica-- 

tion and dissemination, program evaluation and quality assurance, innovative 

programming, inter**program commVinication, interagency cooperation, and cur*- 

riculum standards. A total of 18 specific recommendations concerning these 

areas were made and suggested for adoption by the SE^ 

# 

Sign ifi c ance 

The current study provides a comprehensive description ancl analysis of 

vocational educalj^on services to the handicapped in Pennsylvania~^over a 

three-year period* T!he findings from this study will W helpful in review*- 

Ing state policy, and ][>rocedures with regar;! ^o the vocational handicapped 

program* Accordingly, recommendations were made in li number of areas includ 

ing the following: 

# defining the target population 
. 0 method of funding allocation ^ 



0 
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exemplary program Identification and d^j^ssemination 
use of "best practice'* techniques 



# enhancing loca^ ptogram evaluations 



effecting interagency, cooperation 
exerting a a^^^^ role - 



Study recorameiyfatious also Include procedures for designing and 

operationalizing an on-going tnanagement information system, for handicapped 
♦ 

projects. Thus, the study 'serves not only as aii example of federally 
mandated vocational evaluation, but also provides useful specific findings 
oA the implementation and effects of such projects within a major' state. 
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Abstract - 

Title: Assessment of the Impact of the Bureau of Vocational Educatl>on*s 
^ ' ' Handicapped Program ^ < 

Submitted by : Russell A. Dusewicz, Research for Better Schools 

Philadelphia i Pennsylvania 

Total Funding : $20,000 ' ' ^ 

Beginning and r^nd Dates ; October t5, -igSS-June 30^ 1984 

Summary ; , 

This- .ev^luati-on study represented an effort to pull together a vast . 
array of information and data to describe PDE's vocational eaucation 
•prpgrams for the handicapped from the 1979-80 to" 1981-82 Gchoo'l years and. 
to provide evidence of their impact sQn the target population within the 
state. In, addition, the stiidy addressed the extent to which funded 
projects had developed or improved their prpgram capacity to provide 
quality vocational education programs to handicapped students. ^ 

The conceptual fram6work for th<j study embodied five component si (i) • 
problerasAieeds of the target population; (2) PDE priority solutions; (3) 
PDE-funded activities/programs; (A) program outcomes (impact and ..capacity)-; 
and (5) future directions/policy reGommendatiofts. The study workscope. — - 
involved seven' setjuential interrelated stepcc (1> stfcdy design/liaison 
with PDE; (2) review of existing documentation 'and files; (3) interviews 
with key PDE officials; (4) pilot ^tudy/on-site LEA personal interviews; . ' 
(5) population study/mail survey; (6> sample stu^y/LEA telephone 
interviews; and ^7) analysis/synthesis/reporting of results. • ^ ^ 

The study had several significant outcomes. • It described the 
distribution of over 11 million dolljars f or '334 vocational education , 
•programs for the handicapped oyer the- three-year period under study, and 
established a state-wide computerized data base of all funded projects 
which could be used for future research inquiries. It also took a critical 
loo.k at' the return' on PDE's investment in vocational education for the 
handicapped in terras of the extent of the targiet population served and 
program effectiveness. Finally,- it addres.sed a number of significant . 
problems and issues through a set of specific recommendations 'for\=poli<^y 
and procedural changes at the^state and local levels. Among oth^r areas, 
these recommendations, dealt with: method of funding allocation, exemplary 
program identification, use of "best practice" techniques, enhancing -local 
evaluations, ^fecting interagency tooperation, *and exerting a stronger SEA 
leadership role. V - , ■ . 

Study results .were 'presented in a final evaluation report. Plans for 
dissemination of findings include a brief executive summary distributed to 
relevant audiences throughout the state and presentations at state and 
national conferences* \ • • . 
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I. BACKGROUND 

This section provides an 'overall background for tKe study, including' 
its purpose, a review of related research, and an overview of the balance 
of the present report. 

The purpose of the study is grounded in the Vocational Education Act 
of 1976 (P.L. 94-482). While this legislation made nq' major changes in the 
basic goals of vocational education, it iubstantially changed the 
mechanisms employed to further and enforce these go^ls. Particularly 
noteworthy in this regard are two specific areas where states have been 
given increased responsibility for the^^dellvery of quality vocational 
education. First, states are required to provide^a^^ll^ra^ of 
vocational education opportunities for handicapped learners. "^Pennsyl- 
vania, these opportunities are supported by programs in four areas: 
secondary, postseicondary, guidance, and teacher education. Each year funds 
are alloc;>tfc:d to local education agencies throughout the state to operate 
these programs. A secona noteworthy implication of P.L. 94-482 is the 
requiremeut for states to evaluate all vocational education programs at 
least every five years. All programs are to be evaluated in terrrs of=^ 
planning and operational processes, student achievement, student employment 
success, and the effects of additional services. Moreover, these 
evaluations are to -be designed to revise and improve programs administered 
by the state. To meet this obligation, the Pennsylvania Department of 
Education (PDE) issued a request for proposals to, evaluate the 1979-80, 
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1980-81, and 1981-82 vocational programs for the handicapped. Research for 
Better Schools (RBS) was selected to undertake the evaluation. study . ^ 

In order to meet the evaluation requirements to gather information to 
revise and improve programs, the study examined and analyzed the various 
types of vocational education programs for the handicapped that were 
sponsored by the Pennsylvania Department of Education from the 1980 through 
1982 fiscal years. The study developed an extensive datj^ base to describe 
these programs ..and sought to determine their impact on the handicapped 
population within the state. In addition, the .-study addressed the extent 
to which funded projects had developed or improved their capacity to 
provide quality vocational education programs to the target groups. The 
extent to which program effects were transferred to other sch^iols and the 
ability to continue to provide services with diminished resources was also 
examined. One of the primary outcomes of the evaj.uation study is the 
compviehensive data base developed for all handicapped projects from 
1980-1982 that can be used by FOE to make informed policy decisions in 
continuing program efforts in future years. This is particularly important 
in view of the anticipated diminution in federal and state resources 
available for aJLl educational programs, including vocational education and 
special educatidn. 

Related Research 

To properly understand the current stu.'y and its implications, it must 
' be viewed in t,he context of its historical perspective and related 

researclu This sectior disctusses federal legislation and prior research . 
contributing to this context ♦ 
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Legislation 

In the mid-1970s > educators and coneerned citizens became increasingly 

aware of severe probleml in providing vocational education opportunities 

for handicapped youth (e^g. , Olympus Research •Cprpojr^tidn, 1974; General 

Accounting Office, 1976; Phelps, et al., 1976; Levltari and Taggart, 1977). 

Among the significant problems noted were under-representation of 

handicapped students in regular vocational classr > an over-emphasis on 

» 

# 

prevocational or non-skills training instead of vocational skill training, 
concerns regarding equitable funding procedures, restricted occupational 

offerings, problems with state-level interagency coordination, and the lack 

■■ . 

of professional preparation of vocational educators in serving handicapped 
youth. This awareness set the stage for renewed national interest in 

» 

expanding and improving vocational education opportunities for the 
handicapped* As a result of federal and state legislation, litigation, and 
the efforts of various advocacy groups, existing trends began to reverse* 
Section 504 of the Rehabilitation Act of. 1973 (published in 1977) set 
requirements for non--discrimination on the basis of handicap that specific- 
ally applied to vocational programs. Public Law 94--i42 integrated the 
concept of^vocational education into its commitment to provide *'free and 
appropriate public education** for all handicapped children. The law- also 
required the development of an Individual Education Plan (lEP) that must 
include vocational objectives for all handicapped students in vocational 
education programs. The trend, towards increased vocational opportunities 
continued with the passage of P.L. 94--482 which gave states the primary 
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responsibility for adminlsStering vocational education, setting aside fund- 
ing for t^e handicapped (.^..e.> 10 percent of basic grants), equally - 
matching federal funds with state and local resources, and giving priority 
status to programs for the handicapped in their annual state plans. 

Programs and Needs 

i.jj..Tr -.-•ii- ij I r ^ , 

In response to these mandates and the needs indicated by the cited 

<, 

research* states actively began to remedy^ existing deficiencies. For 
example. Griffin (1978) reported that more than 80 inseryice and preservice 
training programs were initiated in nearly every state to train vocational 
and special education personnel to work with handicapped youth. The 
Pennsylvani^^reau of Vocational and TecWical Education sponsored the 
development of an admini-strators manual" r5"f~plaimtng^r"develop ing, and 
implementing' mainstreaming, self-contained, or cooperative work experi^ce 
programs for special needs learners (Wircenski, Irvin, and Blake, 1981). 
In Pennsylvania's 1980-1981 State Plan for Vocational-Technical Education 
Programs, the provision of me^^ingful vocational education for the 
^handicapped was the "number three" priority for program improvement. The 
state has b^en allocating approximately four million dollars a year^to this 
end. 

The recent report on vocational education by the Secretary of Educa- 
tion to the Congress (United States Department of Education, 1983) indi- 
cates that the enrollment of handicapped students in vocational!, programs 
increased substantially by the 1980-81 school year. These students com- 
prised 3.3 percent of the total vocational educatidn enrollment, a 96 
percent increase over the number seryed in 1975-76. Most of these students 

. ( ■ ■ ■>.■ 



(i.e., 74.7 percent) were served in mainstream vocational programs, with 
largest gains in business education and industrial arts. However, these 
increases must be viewed in the light that handicapped students represent 
9.5- percent of 'the total public school enrollment (grades 9-12) and, as 
such, are still underrepresented -in vocational education despite the 
significant gains* 

Even with in(;;r eased interest, support, and programmatic effort, 
several researchers have indicated that vocational education for the 
handicapped is still in its early stages of development and several needed 
areas of {improvement have been identified. needs assessment by the 
N^itional Association of State Boards of Education (1979) identified several 
remaining deficiencies, including interagency cooperation, personnel 
"preparation, funding methodology, service delivery and program options, 
program evaluation, and services to minority handicapped youth. A study by 
researchers at the University of Illinois and Pennsylvania State University 
recommended improvements regarding involving parents in the vocational 
education process, expanding arid refining administrative re'gulations, 
involving the business and industrial sector, identifying exemplary 
practices, expanding teacher education activities, promoting state \ 
.leadership skills, and coordinating service delivery system^ (Phelps and 
Thornton, 1979). A national survey of individualized education programs 
for handicapped children revealed that only 7.8 percent of all lEPs 
contained short-term Instructional objectives foqusing on prevocational or 
vocational education (Pyecha, 1979). 

In light of these 'needs, it is alarming that funding for Federal 
programs focusing on special needs populations appears to be declining 
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(Phelps^ 1982). Compounding this development, Phelps notes that overall 
enrollments at the secondary level are declining, thereby reducing the 
capability of LEAs to generate funds through typical state aid fonnulas. 
These circumstances have important implications for the identification of 
exemplary, cost'-eff active practices to serve the continuing vocational 
needs of the handicapped population. Systematic evaluations focusing on 
policy issues become vety important. 

Evaluation Requirements ' , y 

Following the passage of P*L. 94-482, several educators and 
researchers developed descriptive guides and procedural manuals to help 
states and localities interpret and implement the mandated evaluation 
requirements. The Education Commission of the States (1979), Foster 
(1979), and Datta (1979)-, described the evaluation roles and 
responsibilities of various groups and illustrated the relationship between 
evaluation, accountability, and planning according to Congressional intent. 
States such as^ Illinois, Ohio* and Vermont sponsored the development of 
guides for evaluating vocational education, several of which specifically 
focused on the handicapped (Albright and Markel, 1982; Wentling, 1978j 
Ghazalah, 1978; and Albright, 1982). Orr (1982) described information ' . 
requirements and data s>urces for assessing vocational education programs* 
These sources "suggest good frameworks for developing evaluation designs, 
illustrate special considerations in evaluating vocational education 
programs for the handicapped, and provide several examples of useful 
checklists and^ instruments that can be adapted in the conduct of 
evaluations th^^t meet mandated requirements. Most of the state guides seem 
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to recommend a self^-assessment approach to evaluation emphasizing qualitative 
data collection supplemented by demographic and fiscal data. Cost-- 
effectiveness models (e.g., Ghazalah) are somewhat difficult to implement. 
^ Discussions of e?cperimental approaches- to evaluation are.*rare. There is 
considerable disagfeei^^^^ of student out c^^ voca- 

tional education, definitions* range from the traditional notion of actual 

, • - ^ ' • .. . 

employment in the field of training to personal success in terms of each 

- individual's specific goals (e^g., in Vermont). Program completion in such 
a case might be the "desired end" of a handicapped student's vocational 
program. For handicapped populations, however, the development of employ- 
ability skills is probably the most frequent and reasonable vocational 
program goal. * . 

While most of the evaluation sources described above are "how to" 
guides, reports of actual evaluation results are scarce. Typically, . 
■ evaluation reports provide^ extensive documentati,on of enrollments and ex- 
penditures but student impact is often ignored. For example, the "Impact 
for 'Handicapped Students" section of the 1982 Congressional Report on 
Vocational Education is limited to^a few brief paragraphs on two exemplary 
programs in Wisconsin and California. Overall, little is currently known 
about the actual impact of the renewed initiative of vocational educatioh 
for the handicapped. 

Organizat ion of the Report 

The report is organized into five main chapters plus an appendix. ^ 
Following this introductory chapter^^ there are additional chapters on 
methods, analysis, conclusions and recommendations, and dissemination. The 
methods chapter details the methodology and procedures utilized in carrying 

'Id 



out the study, .including the evaluation questions, data sources, timeline, 
and limitations of the study. The analysis chapter presents an analysis of 
the data and the . results or, findings of the study. The conclusions and 
recommendations chapter interprets the results for application to PDE 
policies and procedur-es,. and sets' forth recommendations deem^d.^uecessary ' 
for program improvement in keeping with the -id-entif ied staC*5- nee^s and • 
goals. The final chapter on diss-emination describes the plan releasing 
and disseminating the -study results. 
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Presented in this chapter is the methodology used in conducting the 
study. Included are tAe evaluation. questions which served as the focus of 
effort for the «tudy, the data sources used: in obtaining the inforqiation 
needed for analysis, and the timeline within wliich the work was f>erformed. 
A discussion of the limitations of the study is also'' provided. ^ 

f 

Study Quest lons< 

•The. study organized and synthesized existing information on the 

# ..... . . . -■ 

State's program of vocational education for the handicapped and collected 
and analyzed additional pertinent 'information in order to assess' the impact ^ 

of PDE sponsored programs, foj the handicapped for 1979^80 through 1981-82. 

. . • ' ' \ 

T,he study .was designed in accordance with 4 five component conceptual 
framework illustrated, graphically in Figure 1. This figure portrays the 
Interrelationship between the various components .of the study. The 
components themselves are operationally defined' by 'the list of study 
questions f>resented in Figure- 2» Thes.e evaluation questions and ^ 
components represent RBS* interpretation of PDE*s interest in the 
evaluation and constitute the obljectives of the study. All questions are ^ 

i * ' 

addressed in this report. However, the availability of bo.th information 
and study resources posed limitations in some cases. The extensive . list of 
evaluation questions indicates the wide scope of information which it was 
necessary ta collect and synthesize during the course of the study. 
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I» , Probleras /Needs 
of Target 
Population 



7 



!!• ^DE Priority 
Solutions 



III. \PDE-Fuaded 
Activities/ 



Pfograma 



\ 



IV* Project Out- 
coraes 
# impact 
0" capacity 



V, Future Direc-* 
t ions /Policy 
Re conuuenvda t ions 



< 



Figure 1. Conceptual Framework for Evaluation Study of PDE's 
\^ , Vocar.ional Education Prograqi for Handicapped Populations- 
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Figure 2. Summary, of Evaluation Questions, By Siudy Components 



Study Compopen t 

1. Problems and Needs: 



Problems/Needs of 
Target Population 



Ev^jjfat ion* Ques t ion 

i\ How ext^ensivc is|target poptilation 
in Pennsylvania? 

2. V/hat are the special problcn!^/ 
needs of the target population? 



11. Priorities ; 

PDE Priority Solutions 



1. What is the existing PDE approach 
- to solving problems (i.e., stated 
goals, long and short-term 
objectives, guidelines)? 

2^' What are desirable policies/ 
practices for PDE to employ in 
order to accomplish goals? 

3. How were PDE priorities determined 
and validateji? 

4. What are! the relationships (i.e.,^ 
commonalities and unique features) 

, between the various TDE objectives? 

5. po PDE guidelines meet the infor- 
' mat ion needs of local education 

^genci^s? 

^ 6* What is the current PDE funding 
approach?^ 



Programs and Activities : 
• PDE-funded Activities/ 
' Programs ' ( 
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1, What types of activities (i^e*, 
projects) were supported by 
PDE during 1979-80 through 
1981-82, by project category? 

2^ What are the relationships* 
between projects? 

3. What evidence exists that funded 
^ projects represent "best practice" 
in terms of servic'es for handi-. 
capped youth {i*e»i ii^ relation 
to criteria such as lEP character- 
istics, goals/standards, indivi- 
dualization, accessibility, 
. ' responsiveness to job market, 
opportunity, support services, 
personnel prepara'tion, and 
resource utilization)? ' 



Flgiirt 2' (continued) 



Study Component 

III. rrograms and Act tvl ties : 
(continued) 



Evaluation Question 



4. How ar^ funds actually distributed 
'\ ^ relative to the target group and^ . 

"^are the funds, being utilized most 

effectively? V 

5t To 'what extent is there inter-, 
agency cooperation (e*g*, Bureau 
of Vocational Rehabilitation, 
Bureali of Special Education,"* 
Research Coordinating Unit) in 
^ terms of funding and administering 
vocational education projects for 
the haridicappeM? 



IV • Outcome^*: 



Pjroject Outcomes 



1. -Have the needs of the target 
population been met? 

2. Are needs met because of PDE- 
funded projects or other factors? 

3. What is the evidence ot project 
impact fpr each project and for . 
specific types of- projects? 

4* What programs work best for whom? 

5* Have projectjs developed the" 

capacity to continue' toj^meet the 
vocational need^ of handicapped^ . 
students?' r 



V. Policies: 



Future Directions/ 
Policy Recomihfendations 



U What issued or activities re'quire 
• further attention? 

2^ ' What recommendations fof future 
/ , - policies/practices can be made? 



Figure 2 (continued) 



Study Component 

V. pol icies ;- 
(continued) 



\ Evaluation Question 



3. What should be PDE*s role in 
.carrying, but their function ' 
with ^regard to^ . - • 

—V leadership? * < 

— funding? * • ^ 
dissemination of information?. 

— technical ass-istance^/staf f 
development? 

monitoring/evaluation/quality 
,agsurante? ^ 

4. How can PDE improve in carrying 
out it^ current role Regarding 
the dimensions noted aboVe? 

b\ What criteria should PDE employ 
iti funding future programs for 
' the hamiicapped? ' ^ ' 



■V 



o- 



Data Sources 



Four data sources used to obtain the information needed to address the ^ 
study questions. These data sources were: existing documentation and 
files, interviews with key PDE officials, on-site telephone Interviews with 
LEAS, and a mail survey of projects. Each data source is briefly described 
below. ' . * 

Existing Documentatj^on and Files 

A wealth of documentation on vocational education of the handicapped 
is available from PDE. This documentation includes goals, objectives, 
fujiding, procedures, impact, and operations of Vocational education 
programs for the handicapped. Some of the specific documents obtained and 
reviewed for this stud" are listed in figure 3. Such, information served . 
both as an important tool for providing a comprehensive orientation to 
vocational education services for the handicapped in Pennsylvania as well 
as providing direct answers to the evaluation questions posed as part of 

... ^ • 

the study. • ^ 

For each funded -project, PDE maintains a permanent contract file which 
contains the project proposal, audited expenses, and any correspondence 
related to the project. The PDE" staff who 'monitor the various projects 
also keep files which duplicate some of . this information. Staff files are 
discarded after a^few years, or when there is a PDE staffing change. In • 
addition, the PDE vocational offices keep files for their regions (East, 
Central, and West) which contain project proposals and monitoring reports. 
These project files are the most comprehensive data source available, since 
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Figure 3. Existing Documentation Reviewed 



1. Pennsylvania VEA Funds, Estimated Project Expenditure Report (PDE, 
198A).. ^ ' 



'2, Vocational Education Guideliiies for the Application of Federal Funds 
(PDE, 1977, 1978, 1980). 

3. Pennsylvania Vdcational Education Management Information System Report 
(PDE, 1.980, 1981, 1982). 

4-. 19JJ3 Pennsylvania Vocational Education Hearings Summarization (PDE, 
• 1983). 

5. 1983 Pennsylvania Advisory Council on Vocational Educatiort Annual 
Evaluation Report (PACVE, 1983). 

6. , Excerpts from PACVE Annyal Evaluation Reports for 1979, 1980, 1981, 

and 1982. 

7. Pennsylvania State-Plan for Vocational Technical r.aucation Programs 
1983-1987 (PDE, 1982) c 

8. Excerpts from 1980-1981 State Plan (PDE, 1980). 

9. Vocational Education Report by the Secretary of Education to the 
Congress 1982 (USDE, 1983). 

10. Project summaries for handicapped for 1979-80, 1980-81, and 1981-82 
■ (PDE). -V 

11. Vocational Education, State Programs, and Commissioner's Discretionary 
Programs (Final Regulations for Education Amendments of 1976) Federal 
Register, Vol. A2, No. 191, P-53822-53891 , October 3, 1977. 
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all projects are represented. Project files were thus important for 
answering questions which required input from all projects, but they were 
limited to very specific information (e.g.; funding level or project 
objectives), to the exclusion of important areas such as impact on student 
development . 

The three types of files (contract, staff monitor, and field office) 
were\ carefully reviewed, and the- PDE official contract files were selected 
for -use in the study. Using permanent contract files required scanning all 
vocational files to find the handicapped projects^^eded. The 
decentralized fiel4 office files offered no, relative advantage over the PDE 
files, since the monitoring reports, which are kept only in field offices, 
were found to contain no useful evaluative information. 

Project files were assembled toe the handicapped projects funded and 
conducted in the 1979-80, 1980-81, and 1981-82 school years. This vast 
array of material was processed in two ways to add to Its utility. First, 
many of the files were perused to gain familiarity with the projects as 
they are represented by file information. .Then, a project file data base 
was designed to extract key information for further analysis. The data 
base specifications appear in the Appendix. In addition to project 
identification information, the data^ base included coded data on project 
expenditures,- funded activities, objectives, and participants. The data 
base was operationalized on a microcomputer system, 

PDE Interviews 

-Key PDE staff members with responsibilities related to vocational 
education for the handicapped were identified for in-depth interview by 
project staff. These PDE personnel were interviewed using a structured 
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interview guide covering all program issues related to the present study 
, (see Appendix). Items related to the major study questions were /tt^cluded 
* in the interview guide. Figure 4 lists the PbE personnel interviewed, the 



infetirview dates, and the interviewers. 



ri 



LEA Interviews ^ 
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Observations of projects in action and distussions with project 
personnel were built into the study design as the major source of in-depth 
information. The Initial design called for 3 site visits and 2i telephone 
interviews. \k revision increased the interview pool to 31 projects, with 
up to 8 slated fpr sit?e visits. The actual number attempted was later- 
revised to 45, of which 39 (U personal and 27 telephone interviews) were 
completfed>. representing an 87% response rate. The composition of this 
interview sample and the surve^r completions are described in Figure 5. In 
general, projects in the eastern_part_of the state were given preference 
for site visits for logistical reasons. More recent ~proje.cta^ere given , 
priority over older ones to minimize problems due to project staff turnover 
and because the majority of programs funded represented continuation 
efforts. 

These interviews resulted' in a richer, more qualitative data base than 
that of the-i^ail survey questionnaire. For example, the interviews 
explored issues sucjfi as exempiaipy practices, effects on capacity-building, 
and quality of program components through probes and branching questions 
which were not praAical or feasible in. the paper-and-pencil mail survey 
instrument. 

It -wa^^lntende^ that at least ten percent of all funded projects would 
be included in thte • in-depth in.terview sample. It should be noted that 
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Figure 4/ 



PDE Interviews 



Project Type 



2. Handicapped post- 
secondary 



3* Overall 



>DE Staff 



Interview Date 



U , Handicapped secondary Clara Gast:on 



Robert Sheppard 



Garrbll Curtis 



February 15, 1984 
February 15, 1984 



Interviewer 



V Bi ester 
Zeitiln 



February 15, 198A Blester 



4* Overall 



Clarence Dlttenhafer February 15, 1984 



Kershner 



Figure 5 
LEA Interview Completions 



Program Category 



f Percent 

Interviews Attempted \ Interviews Completed Completion 



Secondary 


36 






32 


89% . 


Po8t~Secondary 


5 






3 


60% 


Staff Development 

\ 


3 


V 




3 


100% 


• Guidance 


1 






1 


100% 


Total 


45 






39 


87% 



multi~yeai5.i cai-ryover programs were considered .as single programs and' were 
analyzed as such. Carimjver progr^^ prior ''to the 1979-80 school year w^re j 
not considered. In ordei to insure that the sample was represenj:ative of 



the total group, programs were selected on the basis of certain'^^^lt ratified 
categ,Qries. These included, to the. extent possible, such variables as: 
funding year, program category, geographic region, program size, 
urbanicity, and funding level. 

The final telephone interview reflected the content of model forms 

used in prior studies as well as information gathered in the preliminary 

. .. ■ /; 

steps. of'Jthe current evaluation study^. In addition, questions regarding 

the development jf 'program capacity were incorporated. Sample ^pe^sonal and 

telephone Interviews are provided in the Appendix. 'Guides were developed 

for both personal and telephone interviews, and these also appear in the 

Appendix. Telephone fmierviews^ took approximately one hour each, while 

site visits were given a half-day for interview and observation. In all, 

87 percent were completed successfully. Four additional telephone ^ 

interview sites were contacted, but interviews couldn't be completed 

because relevant project staff were unavailable. 

\ • ■ I 

Mail Survey 

It was believed important, and relatively inexpensive, to obtain at 

least\^iniraal data directly from all participating programs. In order to 

accomplish this', a brief mail survey was administered to all projects not 

scheduled for either telephone or personal interviews that were funded 

i 

' during the study 'time frame. Although the depth of information was 
somewhat limited, data from the mail survey provided a comprehensive 
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overvieW^of some important issues^ The mail survey data were intended to 
supplement existing records and documents* and to provide a broader base 
for interpreting personal and telephone interview data and for generalizing 
* ' to the funded program population as a whole ♦ 

, . Several pre-existing questionnaire forms had ^been used for siMlar " 
purposes by other states and researchers, Ijrcluding those developed by 
Albright (1982), Parker (1979), and Wentling (1978)* These forms had 
Included short answer and rating scale items. Final content for the mail 
survey was determined largely by the initial PDE interviews and pilot study 
site -visits, and items from existing questionnaire forms. A sample mail 
survey form is provided in the Appendix. Self -addressed, stamped envelopes 
were included in the mailing. At least one follow-up mailing was conducted- 
to minimizje non- response as well as several telephone follow-ups. Return 
rates for the mail survey are shcftwf in Figure 6. 



Program Category 



_ Secondary 

Post-Secondary 
Staff Development 

/ 

Guidance 
Total 



Figure 6 
Mail Survey Completions 

Percent 

Surveys Sent - - Surveys -Returned .,- .Campletion 



73 



0 - ■ 



0 



73 



54 



"0 



0 



0 



54 



74'% 



74% 



Si 



\ 



♦ 

Ana lysts Plan 

The study questions and data sources described above provided the 
basic ingredients of the studyVs malys^^^^^ Each question was to be 

answered using the information from one or more data sources as depicted in 
Figure 7. It was Intended that the simplest, most direct answers be 
formulated. Extepsive statistical analyses were not planned. Most 
questions could be addressed by synthesizing Interview results and/or using 
' simple descriptive analyses of project files. 

Broject Timeline 

The proposed timeline of project activities began in October 1983 and 
ended June 1984. Since the project was not actually funded until 
January 1984, some timeline changes were necessary, as indicate.d in the 
Interim Progress Report (April 15, 1985)'. The revised timeline specified 
January and February for planning and data collection instrument develop- 
ment, March through May for data collection, and June for analysis and 
reporting. This schedule was followed in practice, with minor deviations 

which did not hamper the timeliness of the study. ^ 

, . . ^ • . . . 

Limitations of the St^dy 

There were several factors which did limit the study ^s^ffectlveness 
• and should be kept in mind while considering the findings and recommenda- 
tions presented in the following two chapters. First, the r^s<^rces avail- 
able for conducting the study were quite modest — • less than $20,000. This 
' level of resources constrained the person-time that could 'be devoted to ^ny 
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Figure . 7 

% Analysis Plan and Data Sources 



Evaluation Questions 



,1. Problems, and Needs 

1. Bxt.ensiveness of target 
population 
, 2. Special needs/problems 



II. Priorities 

1. Existing problem-solving 

approach ^ 
1; Desirable policies/ 

practices 

3. Determination of 
priorities, 

4. Relationships 

5. Adequacy of funding 
guidelines 

6. - Current funding approach 



/ 111. Activities and Programs 
I. Types of activities 
supported 
, 2. Project relationships 
3. Extent df "best practice" 
4., Distribution' and use of 
funds 

5. Interagencjr cooperation 



Is' 



4 



4^ 



Data Sources 



Existing 
Documentation 
and Files ^ ^ 



/ 
/ 



PDE 
Interviews 



/ 



?A'..r---,,,.-,y.. 



24 



37 



LEA 
Interviews 



Mail 
Survey 



/ 



/ 



* 



, . , Figure 7 (continued) 

Analysis Plan and Data Sources, 




• 4 


* 

Evaluation tjues^iorts 

' i I ' v ' 


Data Sources 


Existing 
Documentation' 
and Files 


' " • " "V 

PDE ^ 
Interviews 


LEA 
Interviews 


Mail 
Survey 


\ 

IV • Outcomes 










1. Extent of needs / , 




/ 


/ 




satisfaction 










2. Funded projects vs* 






/ 


/ * 


" other factors 










3# Evidence of project 


V 




/ 


/ 


impact 










4 • ^Program effectiveness 






/ 


• /; 


5* Extent of capacity- 








/ 


^ building 
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Policies 








!• Issues for further 




J 

Y 


V 


V 


attention 










2» Reconimendat ions for 


* 


/ 


/ 


y 


future Dolicies 










# riAcv s xucure roxe ano 




/ 


/ 


/ 


functions 










• 4» PDE role improvements 




/ 




/ 


5* Suggested future 




/ 


/. .. 


- / 


funding criteria 




4 
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/ ^ 
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project task. Sedond, in the face of modest resources, there ^were a large 

number of handicapped projects — 334^ These were spread across three 

/ 

fiscal years and three project types; thus, nine .categories of projects . 

wertf; included in th6i_study^ Each . categgry _^ theoretically could have been 

treated separately, but practicality would not permit it. Third, the 
projects were conducted from 1979 to 1982, while the evaluation field work 
took place in 1984. This titue difference undoubtedly affected the validity 
of , the interviei^ responses* Those interviewed, may have had. difficulty in 
retrospectively distinguishing the project year under study, or worse, the 
most appropriate interviewees may have left the 'school district by the time 
the evaluation took place. 'Finally* in light of .the limited study 
resources, some dependence on already existing data was necessary. PDE was 

■ . ■ . \ ' 

able to supply data which met this need for may of the study questions, but 
one ^rea was found to be deficient — program outcomes* Outcome* or 
impact* information was rarely available on individual projects. This 
limited the analyses and conclusions which could be conducted and drawn in 
this area. 

4 • ' 

♦ 

^ I 

<^ 

«/ 

I 



, _ III. ANALYSIS ' . •■• ^ 

' V • • * 

♦ • 

The analysis chapter reports ther results^^.of the evaluation study. 
Major- findings are organized and discussed in accordance with t)i«^ framework 
of evaiuatiort""questions preserfted in the previous chapter (see Figures. 1 ' 
and 2)». The evaluation questions, arid hence the sections whicll' comprise, 
this chapt<^-r, are grouped' into five general areas: • problems and needs, 
priorities, activities and programs, outcomes, and policies* Evaluafion- 
questions are addressed within each grouping as completely and as concisely 
as possible • Answers to these questions are in 'some^ cases necessarily ^ 
limited by the quality and completeness of the data available. 

• " - - ' . •* " . . • 

a Problems and Needs ' . , 

N »^^. M I« •n^'wm^'^^m^* ■» >..f / j 

- r.- , • . 

'The problems and needs of the target population • set the 6t:age for the 
balance of the evaluation or study questions* 

Question 1 ♦ How extensive is the target population in Pennsylvania ? 

• • • • ( • ' 

Any question regarding the extent of the target population in 
Pennsylvania is difficult to answer because each of the different data 
sources available* give widely disparate estimates* Nationally, handicapped 
students represent ^about 9*5 percent of the secondary school population, 
based on 1981 statistics^ This percentage X^ reported to be ri§ing; At 
the end of the^ 1982 school year; the percen^tage reported in Pennsylvania 
was between 12% and 13%* - " ' . 

7 • ■ 

' To define the target population operationally as that group of . 
handicapped students for whom ^vocational* education is appropriate still; ^ ' 
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permits several interpretations. One would be t'hat the number of 
handicapped students in vocational education should approximate the 
proportionate number in^the total population. A second would suggest that 
vocational ^e^ducation is particularly appropriate for the handicapped who . 
are less likely to go on to college, and thus vocational education and 
concrete learning are good ways to teach abstract skills to this group. 
Following this- line of reasoning, the 'number of handicapped students should 
exceed the proportionate number in the total population. A third 
interpretation represents the pervasive attitude among vocational educators 
that handicapped students are less likely to get into programs involving 
physical requirements, so fewet handicapped students sh'/uld be. expected in 
vocational education. PDE, of course, has no official policy with regard 
to any of- these interpretations. 

Different reports indicate that between 11,000 and 18,000 handicapped 
students participated in vocatioiial education in 1981-82. The former 
figure is deri^d^from the accountability report of the State Advisory 
Council for Vocational Education in its accountability report, while the 
latter figure comes from the Vocational Education Management Information 
' System '(VEMIS). If the figure of 18,000 is used, this represents a 55% 
increase in- the number '^of hai.dicapped students in vocational education over 
the four year period from 1978-1982. This figure also represents about 
3.9% of the the total secondary school population, and about 31% of the 
handicapped population, based on an estimate of 55,000 handicapped 

secondary school students. About 38.6 percent of regular students 

r 

participate in vocational education. Handicapped students in 1981-82 

represented 10% of the total vocational education enrollment. This 
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compares favorably with the A. 3% national figure. Still, assuming these 
figures are correct, the handicapped are slightly under-represented ^iven 
expectations* ^ ■ 

% 

It should be noted that no attempt has been made in any of these 
estimates to separate the number of handicapped served by federal vs. 

A 

non-federal supported programs. Undoubtedly these percentages would be 
greatly affected by such an analysis. 

Wlio are the participants in the vocational education programs for the 
handicapped? This question addresses another aspect of the extent of the 
target population in Pennsylvania. Based on estimates from VEMIS, the 
make-up of the vocational education handicapped population is as follows: 
25% learning disabled, 10% SED (socially and emotionally disturbed), 59% 
mentally retarded (both educable and trainable), and 6% physically 
handicapped. About Al% are reported to be in mains t reamed programs, while 
59% are enrolled in separate programs. These estimates are partially 
corroborated by results of the present study* s mail survey as indicated in 
Table 1. This table shows percentages of programs serving various 
handicapping conditions and percentages of students from each category 
served. 

Data from site visits and interviews reveal that in many cases the 
handicapped vocational education enrollment is determined at local sites 
through the lEP process* Several sites indicated that they served all 
handicapped students referred to them by study teams. Others noted 
limitations regarding the number of slots available to handicapped 
students. Still others indicated^ that th^ students themselves and the 
study team decide whether or not vocational education is appropriate. 

29 
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Table 1 



Program Services by Han^capp,.rig Condition 



Percent of 



Percent of 



Handicapping Condition 


Programs Serving 


Students Served 


EMR . ' I 


74% 


46% 


TMR 


28% 


13% 


LD . 


54% 


28% 

J- 


SED 


35% 


10% 


OtUer 

4 

™ _ 


31% 


3% 
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Question 2> What are the Special Problems/Needs of the Target Population? 

Project proposals* on--slte and telephone Interviews > PDE iaterviews^ 
and other dod^ents served as data sources for addressing this question* 

Prior to recent federal and stat^- legislation, handicapped students 
traditionally were actively discouraged from partifipating 1q_ vocational 
education (see enrollment data) • Thus, initially, sat^aside funds were 
needed to support such prografms. Now the overarching state goal is to 
mainstream students whenever possible* A major need identified in 
connection with this goal is to enable such mainstreaming to occur* 
Related to this is a need for support services to improve the mainstreaming 
process* Significant staff development is also, needed for regular 

V, 

vocational teachers with respect to handicapped student background . 
Information, attitudes, and so forth* Handicapped students need more 
special support and individual attention beyond regular programs** This 
would include special counseling, remedial tutoring, more time to learn, 
and in-shop support* Several, people interviewed at the local level 
indicated *-l:l^at--handicapped students, in general, cannot survive in regular 
vocational programs without this extra support* Also indicated as needs 
were factors related to curriculum methodology, for example, 
individualization, task analysis, and remedial programs* Teacher needs 
were thor^ht to be substantial* Teachers need to be creative, flexible, 
and adaptable* Attitudes are also a big problem, both at the classroom 
level and at the administrative level* 

Several sites Indicated that a major need exists for pre-vocational 
programs, where students can develop appropriate awareness, attitudes, and 
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habits regarding the world of work* These kinds of /programs are not 
encouraged under current state regulations* Other sites noted needs with 
* regard to co-op programs. These included job dey,elopment> placement, and 
follow-up* It was noted that while sites can train a handicapped student 
with job skills, they need, additionally, to work with .employers with 

regard to attitudes and expectations* There is also a need for follow-up 

/ 

with graduates once they are placed. 

Work experience opportunities are also a major need, as with regular 
vocational education programs* However,, handicapped students especially 
need hands^-on work experience, either through co«-op or in a shop* There 
are also needs with regard to employment, drop-outs, and attendance as with 
regular students* Vocational programs need to be more motivating* 
Accesisible facilities also represent a major need for the handicapped as 
well as appropriate equipment and adaptations* 

Finally', many of those interviewed indicated a major need also exists 
in terms of the dollars available to support a vocational education program 
for the handicapped*, 

Priorities 

'\ 

Facing the identified problems and needs of the target population, the 
PDE has assigned priorit^ies to certain solutions* These are represented in 
State policy with regard to vocational education, ^ttate regulations, and 
funding guidelines* Study questions relating to these priority solutions 
are discussed below* ^ 

s 



Questlcns 1 and 2. What is the existing PDE approach to solving problems? 
Wha^t; are desir^i^^ a^^^l practices fof PDE? 

< 

?^ Vocational education projects fof the handicapped over the years have 

addressed the state goals for vocational education (see Appendix), as have 
other funded vocational projects. Table 2 shows the extent to which such 
projects have been committed to each of the state goals during each of the 
years reviewed in this evaluation study. As can be seen from the table, 
the overwhelming'/ majority of programs are grouped under goal #8, expanding 
Opportunities for the disadvantaged „and handicapped • 

The PDE approach to solving problems of vocational education for the 
handicapped may be considered to have been documented in the goals and 
'subgoals for funding for handicapped vocational education programs. While 

V 

these statements provide some indication of the PDE approach, in~depth 
;w ' interviews with key PDE officials and local program staff provide a more 
analytic view of this approach. From this latter viewpoint, it would 
appisar that the PDE approach is more fiscal than programmatic. One person 
^ interviewed indicated that "they throw money at a problem in order to 

, satisfy the advocates." Another claimed that all programmatic decisions 
are made at the local level and not by the state: "local people define the 
problems and solutions, but they don't have the capacity to deal with the 
problems without -federal and state dollars." Some of those interviewed 
felt that the state currently has no real comm,itment to vocational 
education for the handicapped: "The dollars supporting these programs are 
federal, matcheci by LEA funds, and PDE simply monitors' the federal 
. mandat65»" Moreover, the PDE approach appears to come from funding 

categories in the legislation. That is, "handlcapped/disadvantaged" exists 
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Table 2^^' ' 
Projects and State VocationaJ^j^yGoals 

\ 

t 


\ 


Projects 


1979-80 


198.0-81 J 


1981" 


-82 j 


Total 


\ 


\ Goals 


# % 


// % 


# % 


// 


% 




t » t IXI JII1^I1I»""" II" illB l 

t 

: I. Vocational skill 
development 


' 10 9% 


4 


3% 


8 


6% 


22 


6% 




!!♦ Adult^ and post secondary 


5 A% 


5 


U% 


% 


3% 


14 


3% 




Alternative programs 


3 3% 


7 


5% 


9 ' 


7% 


19 


5% 


"•; y 


IV. Articulation across levels 


* 

2 2% 


2 


1% 


0 


0% 


4- 


V 

it 


^>C ' 

{■■/■-.■ 


•5 1 
* > 

V» Understanding educat4.on 
and caree;r options/ 
placement 


5 , A% 


1 


1% 


0 


0% 


6 

# 


<• 

2% 


Sj- ■ '. 


VI ♦ Equal educational 
•opportunities 


2 2% 


1 


1% 


0 


0% 


6 


2% 


i'. ■ ■ ^ 

»^ 


• 

VII ♦ Professional inserv^ce 
programs 


0 0% 


1 


1% 


3 


2% 


4 


1% 




VIII* Expanded opportunities 

for disadvantaged/ 

handicapped 
• • 


84 . 76%. 


113 


84% 


. ii2 


82% 


^09 

1 • 
1 


81% . 

> 














\'.v . 

- 1'- 


.■,>H>M.'. ».W..,*^4.>.«M*.>.^)t..i i ■ 

TOTAL 


Ill 100% 


134 


100% 


136 


100% 


381* 


100% 


1 


* Most projects had more than one goal. 


• 




1 


T 
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^ as a funding categoryV so PDE.has a ^^handicapped/disadvantaged" person on 
i^taff to'^oordinate and monitor it. View this in contrast with a potential 
alternative approach of assigning responsibility to operational staff; 
that is, permit. areas of the department which relate to special education 
to monitor vocational education 'grants for the handicapped and Chapter 1 
staff to monitor grants for the disadvantaged. . Similarly, in its current 
configuration, handicapped vocational education la divided up further into 
post-secondary, guidance, disadvantaged, and secondary (paralleling the 
funding categories), all with different people responsible for them. 

Prior to federal and state^ legi^atlon in 1976-77, most of the 
vocational programs for the handicapped consisted of pre-vocational 
activities (e.g., assessment, work adjustment). After such legislation, 
the emphasis moved toward skili training and mainstreaming of handicapped 
students into regular vocational programs. According to current 
guidelines, vocational programs are not supposed to support pre-yocational 
activities, though many still exist and many more wish that continued 
support for pre-vocational activities would be forthcoming. PDE has not 
changed its basic position since 1979-80.' That is, the LEAs primarily set 
the priorities f oi^ the vocational education programs for the handicapped. 

There currently exists a strong feeling among PDE staff that the PDE 

approach should be of a capacity building nature. Tliat is, funds should 

used to build local programs towards a capacity for mainstreaming. Given 

this orientation, there are two principal ways to support it: a 

consumptive approach and a capacity building approach. Following a 

consumptive approach, funding would be made available for things like 

* 

perpetual staff development, and ongoing program operations. Capacity 
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building would require attention to teacher attitudes, facilities, 
upgrading teacher skills, and materials and methods. Basied on the 
assumption that capacity is finite and can be reached, this may be the area 
to focus attention on initially* To some extent, LEAs have used program 
monies for capacity building, particularly with regard to facilities, 
although mo^^funding is spent on program opera^ons* 



PDE/s role with^respect to the funded programs is usually one of 



problem solver of compliance or fiscal concerns, rather than programmatic 
concerns. There is a strong feeling that the current PDE role should be 
more in the area of technical assistance, emphasizing programmatic ^ 
concerns. PDE also focuses .almost exclusively on compliance rattier than 
quality assurance ♦ A strong feeling exists that PDE should focus more on 
assuring quality in the operation of local 'programs. Moreover, monitoring 
reports compiled by PDE and regional offices contain little useful 
information on local programs. While there may be utility in having, 
monitors visit and regularly communicate with local programs, documentation 
on, these visits and communications is of little v^lue in its present form. 
One interviewee felt that compliance visits were only useful as a social 
function. 

Although t]tie state funds evaluations of programs, these evaluations at 
the local levtl are not in-depth» In addition, final reports are not 
r^g;iired of projects* This lack of final reporting fosters a lack of 
accountability. Some LEAs assume that the dollars will always be there, so 
they don^t have to be accountable, according to some interviewees. Many 
interviewees recommended instituting substantive reporting requirements. • 
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Other recommendations stemming from PDE and LEA interviews included 
specification of a state role in dissemination of Information on vocational 
education for the handicapped, identification and publicity for exemplary 
programs and practices, and instituting networking among the projects which 
have been Established. 

Questions 3 and 4. How are prio rities deter mined and validated? What afe 
the relationships? 

As noted previously, the state education department disposes of 
federal funds for vocational education of the handicapped, the locals set 
priorities, and the school boards ^eek or want funding from the state. The 
state plan outlines the types of activities that can be supported and PDE 
doesn't add to or change the federal priorities. One respondent to the 
interviews said "It doesn*t make any sense to prioritize goals at. the state 
level, since, with entitlement; it doesn't matter anyway," ^ . , 

A new state plan is developed each year. Program people provide input 
to it but .have a relatively minor rol« in its contents.^ The plan proceeds 
througli^^ff to section heads to division chiefs to the bureau director. 
The Staq^^^^ry Council for* Vocational Education has^'a role to adv|s€, 
IC submits a^^anCial report of recommendations. This= state advisory \ 
council usuallf^doesn' t recommend on handicapped issues each year, although 
there were recommendations during years included in the current study. For 
example, the state advisory council recommended a special education 
vocational education coordinator, and had several other recommendations 
during the 1982-83 program year* I;i addition, the Pennsylvania Association 
for Re tarried Children (PARC) and the Pennsylvania Association for 
Vocational Education for Special Needs Pupils (PAVESNP) provide testimony 
at hearings, and that is taken into account in developing the state plan. 
Civil rights, reports a^re also taken into actiount. ^ 



Questions 5 and 6, Do' PDE. guid elines meet the Information needs of local 
education agencies? What Is the current PDE funding approach? 

t 

Based on Interview data, the' consensus from local programs Is that 
information needs regarding applications for funding are being met, and 
that locals are familiar enough with the application process to be less 
demanding in this regard. Locals- can request help from regional offices 
and they do so when needed. The general perception is that the regional. • 
offices provide satisfactory lielp with regard to requests for aid from the 
lotals. There is a perception on the part of some state department of 
education staff that LEAs request help more frequently than desirable, and 
they fear that this may encourage dependency. 

As far as the information needs of locals in other areas is concerned, 
the Information coming from PDE was not felt to be sufficient. For 
exan^le, data collection and MIS requirements were perceived by the locals 
as generally not" clear. These requirements appea^r to locals to change too 
frequently and the reports based on such information are viewed by them as 
being hard to interpret and use, too late, and containing the wrong types 
of information ,to be reported to programs. Local programs want good and 
up-to-viat^: Information, and would like to see useful evaluation data. 
Local program^ also report that they need information on promising ^ 
practices. They feel that there is nd dissemination network or sharing 
among funded programs in this regard. Several local program |)ersonnel 
suggested that they would like to see an annual meeting or two to discuss 
viable strategies for meeting peculiar local situations and needs in this 
and related areas. 
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Some problems were reported by locals with respe^it to turnover in'l.EA^ 
staff assigned as federal coordinators, anXthe effect of this upon 
operation of local programs. - " 

With regard to technical assistance provided from regional and central 
-.office .program personnel, the perception of LEAs is that the quality of 
assistance varies from person to person. Some personnel are extremely 
helpful and some are not. 

With respect to the PDE funding approach, PDE and LEA interviews 

f 

indicate that many believe there should be more direction with regard to 
the PDE funding approach, and pjore clearly defined priorities. Such PDE 
funding should be aimed at capacity building, so that if ^ the dollars 
received for support 'of such programs are cut, then the LEAs can deal with 
the problems i^,ased on capacity that has already been developed and caa, 

^^remove themselves in many ways from dependence upon the receipt of extern^il 

\ • 
fui^ds* 

\^ . 

^^VpK does. not have leeway with regard to competitive funding for 
support of vocational education for the handicapped, except in teacher 
education and research and evaluation. The allocation approach, rather 
than the project method, is required by legislation. The perceived 
advantage of the allocation method is that all schools can participcite. 
Schools d,on't have to be particularly aggressive or sophisticated with 
regard to proposal preparation in order to receive funding. This prevents . 
politically influential districts from getting all of the funding dollars. 
Such a broad based funding approach has much to recommend it. The 
disadvantage of the allocation method is that funding and project 
development can become cafeual over time, due to lack of^'ccountabillty and 
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lack of incentives for developing and operating quality programs. The 

allocation method takes something away from the state leadership role in 

c 

this ar^a» some interviewees felt. * 

The project method h^^J>e6h tried with other components. Several 
problems exist with this method. Initially, there are often an 
insufficient number of applicants. This method creaJLes a lot of paperwork 
and effort surrounding the proposal review process* 

Much negative feedback was received from the local people interviewed 
regarding the funding approach* particularly with regard to the timing. 
Many of those interviewed claimed the funding is received too late* and 
that they don^t know if they will be continued from year to year or the 
amount, of funding upon whlch^ they can depend* There were also complaints 
that there were not enough funds for placement and for follow-up i 
aptivities* There were also perceptions of inequitable funding, with too 
large a share of the funding going to urban areas. Some alternative 
suggestions were offered with regard to changing the funding to a two year 
cycle « 

The Vocational Education, Act affects the amount o^ money^ and the 

approach to funding* For" entitlement formulas this involves criteria such- 

♦ .. " 

as the percent of economically depressed-^ high unemployment, emerging 

manpower needs, relative number of low income families, and relative 

financial ability of the district* 

PDE staff interviewed indicated tha,t they did't like the funding 

- ' ^ '^'^^ 

■ ■ m- 

.approach; either, because it doesn^t appear to foster quality. They would 
rather see the funding as a competitive process. They recognized that the 
distribution of funding might not be viewed as equitable if it were 
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•determined on the basis" of ' quality, but felt that the way to insiure quality 

±q' to institute„,corapet:itive funding as well as aq count ability and' 

. ■ ' f.' 

monitoring. They believed that, this would lead to a balance between equity 

and quality^ ^, * • 

The qualifications for funding for monies that are part of "the 10% 

set-aside i^nclucffe the following* Each student much have an'IEP* Each 

student must be placed , in the least restrictive environment* Each student 

must^have equ^l^access* to regular vocational education* Thete must 'be , v 

. cooperatix?n between vocational educd' lon and special education ^taff..' The 

handicapped must be diagnosed by a qualified physician and/or psychologist. 

Funding m»:st be used as excess costs; that is> costs over and above that of 

the regular program. There must be assurances that handicapped .students . 

cannot otherwise succeed in a regular school ptogram. In addition, the 

dollars must be used"*for_ vocational instruction i or services to enable * 

success in a regulajr vocational program. Pre~vo'cational prograrns or 

activities are non appropriate. Bas6d on site visit;s and interviews at .the 

local level, for* the moat part, compliance was observed with the above 

qualifications, though often at a superficial level. ' ^ 

». * 
\ ■ ■ ... ' 

Activities and Programs 

The study questions addressed in this general area involve the types 

of activitie^s and characteristics of programs , funded by the Bur(a^u of 

Vocational 'Education under the vocatiof^al education program for the 

handicapped • * ^ , 
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Questions 1 and 2. What types' of activities are fun ded by PDE? What are 
the relation ships between 'projects? 

The types of activities and the type of orientation that a particular 

project Jias is determined largely by the category in which it is funded* 

All projects for tUe three fiscal years under the study were identified 

from the PDE contract files and sorted under program category ♦ The number 

of projects funded by category and by fiscal year are presented in Table 3* 

As can , be seen in the table, a total of 334 projects were identified, for » 

the three fiscal years. The vast .majority of these programs were funded in 

the category of ^^handicapped secondary.'* These projects are funded through 

entitlement, or through the allocation method.'. They constitute 

approximately 82% of the total number of projects funded over the 

three-year period. In addition, they represent an Increasing proportion of 

the total number oi: projects being funded across" each of the three ye'a^s. 

Table 4 shov;s the numbers of vnew projects by program funding - 

categoYy, funded for each of the .fiscal years under study. As can be seen 

in the table, a total of 78 new projects were ^funded during the three--year 

period. A*s sucl), Table 4 represents a subset of the total number of 

projects presented in Table 3. Again, the great majority of prograuis fall 

into the '^handicapped secondary" category. A total of 23% of the total 

number of projects funded over the three-year period ' represented new 

projects. .This indicates that the largest proportion of projects being 

funded each year are continuation projects, 

'Data were gathered on the goals of the projects and the types of 

• activities that they represent by means of a mall survey, and these data 

are depicted in Tables 5 and 6. Table 5 indicates that the largest number 

of projects reported one of their p]t;imary gpals to be the "acquisition of 

Vocational skiXls/competencles . " Only 4% of the programs indicated "staff 
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Table 3 



Projects Funded by Program Category 



Program Category 



1980 



1981 



1982 



Total 



2-2 H/C Secondary^ 



68 



. 100 



AOS, 



273 



2-1 H/C Post Secondary 



10 



14 



13 



37 



3^5 H/C Guidance 



10 



18 



3->6 H/C~Currl<iulura 



3-7 n/C Staff Development 



0 



0 



0 



0 



0 



Total 



88 



124 



122 



334 
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Table 3 



Projects Funded by Program Category 



Program Category 



1980 



J 

t-1 H/C Secondary 



2-1 H/C Post Secondary 



3-5 H/C Guidance 



68 
10 

10 



. 1981 



1982 



100 



14 



105 



13 



Total . 



273 



37 



18 



3-6 H/C Curriculum 



0 



0 



3-7 H/C Staff Development 



0 



1/ 



Total 



88 



124 



122 



334 



/ 
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Table 3 

Projects Funded by Program Category 

1 



Program Category 



1980 1981 1982 Total 



2-2 H/C Secondary 68 ^ 100 105 273 

2- 1 H/C Post Secondary 10 14 13 37 

3- 5' H/C Guidance 10 6 2 18 
3-6 H/C Curriculum 0 0 0 0 
3-7 H/C Staff Development 0 4 2 6 



Total 



88 124 122 334 
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Table 3 



Projects Funded by. Program Category 

I 

Program Category . 1980 1981 1982 



r 



2- 2 H/C Secondary 68 100 ^ 105 
Zyl H/C Post Secondary 10 14 13 

3- 5 H/C Guidance 10 ' ^ '6 2 
3-6 H/C Curriculum 0 0 0 
3t7 H/C- Staff Development 0 ' 4 2 

, .^ . . ^. , . ■ I .1 I . . ." 

Total • 88 ■ ' ' 124 122 



*W* .H I M I.. I, 





^ 






Table 4 ' 










v • ' ' * ^ , New Projects. Funded by Program Category 


i 


optogram Category ' 1980 

\ 


^" ^1981 19S'2 ^ 


t 

Total' 


1 

2-2 H/C Secondary , 23 


,9 ' 24 


56 ' 


2~1 H/C Post Secondary 7 


3 ' *9 


-,19 - 


3-5 i^C Guidance * 1 


2 0 


3 


3-6 H/C Curriculum , 0 


. 0 0 ^, 


0 


3-7 H/C Staff Development 0 

— — ^ . 1- , , ( :•, ■ 






0 0 


'0 

» 


Total , 31 


1 

14 33. 


78 


^. 


/ 
/ 




* 






r 


/ 

f 


\ 




4 *^ 


* 

/ 




i 

V 
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Table 5 

Project Primary Goals, >^ 
> 1 ' 

Goals Percent of Projects 



1. Acquisition of Vocational Skills/Competencies ^ 81% 

\ • . 

2. Mainstream into "Regular" Vocational Classes'^-.. 65% 

3. Development of Pre~Employment Skills 57% 

4. Placement in Full/Part Time Job 

. » • 1. 

5. Staff Development , 



4% 



6. Other • ' * 20% 
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Table 6 



Project Activities 



Activities 



Percent of Projects 



. 1. Specific Job Skill Training 

2. Individualized Program 

3. General Vocational Skills 

4. Pre-Emplo5rment Servicea. 

5. Competency-Based Instruction 

6. Counseling 

7. Mainst reamed Program 

8. Remedial Training 

9. Evaluation/Assessment 

10. Work Experience/Coop. 

11. Self-r-Contained Program 

12. placement Services 

13. Staff Development 

14. Other 



65% 
61% 
54% 
48% 
46% 
46% 
46% 



.43% 
37% 
2^% 
24% 
11% 
24% 



v 



development" as a prijiaary* goal'. Table 6 suggests that "specific job skill 
training" appeared as a principal project activity in 65% of the programs. 
This represents the most frequent activity reported by programs through the 
survey. "Individualized program" was reported as a principal activity by 
61% of the- programs. "General vocational skills" and "pre-employment 
skills" ranked third and fourth in frequency amon^ the projects surveyed, 
respectively.- Only 11% of the projects indicated "staff development" as an 
activity. This agrees generally with the findings for primary project 
goals presented earlier in Tab-le 5. ■■ The finding of "specific job skill 
training" as the most frequent activity among projects coincides with the 
large percentage of programs indicating "acquisition of vocational 
skills/competencies" as their primary goal, as reH^cted^in Table 5. 
Cutrent mandates are reflected in the finding that "mainst reamed program" 
was represented in 46% of the programs as compared to "self contained 
S^^^i. program" which was only represented in 28% of the programs* Moreover, many 
of the self-contained programs included components which involved 
ma.lnst reaming* 

Qu estion 3t To what extent do projects represent "best practice" in 
f services t o the handicapped? 

"Best practice" is a subjective ter.m which represents a judgement as ^ 
to the degree to which the procedures or methods which are used represent 
the ones thought to be the most effective* For purposes of this study, a 

i 

vocational program for the handicapped exemplifies "best practice" when it 
incorporates the following program features: 
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# individualized instruction 

# integrated instruction (mainstream) J 
' ♦ real job skills (job skills and training) 

# placement in paid employment \> 

# competency-based Instruction 

# community-based instruction (work experience/co-op) . 

While it is not claimed that the above program features represent a 

comprehensive, list of those that might be considered, "best practice," they 

k < 
do represent v^hat is commonly expected as non-mand/ited optional features of 

^ ... ■ 

programs that wou\d be illustrative of "best practice.". To determine 

whether the funded projects represented "best practice," an analysis was 

performed on the data collected in the mail survey and through the 

personal/telephone interviews with LEAs. In analyzing these data, the 

number of "best practice" features was totaled for each project and a mean. 

was computed across projects for the mail survey and the interviews. Table 

7 shows the results. In this table, the mean number of "best practice" 

. . •■ , 

features Is. shown separately for each type of survey with the corresponding 

percentag.es based on the total numbeir' of features possible to exhibit 

(i.e., 6). Thus, it can be seeii that approximately half of the features of 

what might be considered '"best practice^' are exhibited in current 

vocational education programs for the handicapped. 

A slightly different perspective on "best practice" might be obtained 

by looking at the content of the programs funded. That is, the extent to 

which the programs focus on vocational skills vs. pre-vocational skills can 

be examined. For this purpose, both the mail survey and interview data 

were analyzed once again. The analysis consisted of computing the 

percentage of programs which indicated that they taught vocational skills, 
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Table 7 ' 
'*Best Practice" Features Exhibited" 



Data Sources 



I^umber of Features* Percent 



Mail Survey . 




f 


2.93 


V 


49% 


Interviews 

r 






3.13 




5Z% 



My 



*Highest possible number of features - 6. 
1 
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Table 8 



Types 6f Skill's Taught 



Skill Types 



Mall Survey 



Interviews 



Vocational Skills 



3i% 



^ 18% 



Pre~Vocational Skills 



19% 



13% 



Both 



44% 



62% 



Neither 



4% ■ 



8% 
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Table 9 




• 


•Funds Distributed by Category 


Category 

• 


1980 • 19.81 


1982 Total 


♦2-2 H/C Secondary 


$2,615,115 (81%) . $3,042,866 (77%) $3,458,883 (86%) $9,116,864 (81%) 


2-1 U/C Post- 
Secondary' 


401,683 (12%) 780,509 X12%) 


'390,361 (10%) 1,272,553 (11%) 


3-5 H/C Guidance 


224,050 ( 7%) 329,018 ( 8%) 


106,597 ( 3%) 659,665 \( .6%) 


3-6 H/C Curricv i m 


0' ( 0%) , ( 0%) 


-0(0%) ' e (* 0%) ■ 


.3-7 H/C Staff 
- Development • 'I 


• 0 ( 0%r- , 98,180 (Vi) 

■ ft. 

, \.„ ; ■. 


65,254 ( 1%) 163,434 ( 2%) 

^' . .■ ■' 


Totals 


$3,240,848 -$3,950,573' 

•< 


$4,021-,095 , $11,212,516 ' 
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increases in Overall funding across the three fiscal years* In addition, 
it should be noted that the ^'handicapped secondary**^ level i^epresents funds 
distributed by enfritlement. 

When project distribution is examined by region (see Table 10), it can 
be seen that, over the period under study, the largest percentage of 
proj'ects was. funded in the western region. The number of 'projects funded, 
however, does not necessarily reflect the amount of funding represented by 
such proj'ects. 

Question 5. To what extent is there interagency cooperation in funding and 
administering the program? - ^ ~ ^ 

This question represents. a key issue in the literature^ In 1979--80, 

the. State Advisoijy Council for Vocational Education recommended that 

interagency cooperation be improved In this area. This was reflected in 

the state's -annual plant Despite these policy initiatives to improve 

Interagency cooperation, staff interviewed at the st^te level generally 

reported a suspicion ,that there wds not a whole lot of "interagency * 

cooperation present in -the funding and administering of the program* It 

was felt that ''since the funding doesn't require it, there is \\o incentive 

to do it." State staff also Indicated that "there may be -more cooperation^ 

at the Uik level" , than at -the- SEA level. More interagency cooperation was 

observed at the LEA level during' on-site. observations and interviews than 

was found ^it the SEA level. This may be-a result of LEA priorities and 

■ H ' ' ^ • ■ ' - 

funded activities. . . 

Vocational rehabilitation was Involved somewhat at the loc^l leVel 
with post-secondary project^* For example, at one local project they were 
able to use vocational rehabilitation fui is for ^ome job development and 

\ 
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follow'-up Activities that would not have been possible with Bureau of 
Vocational Education funding. 

Bureau of Vocational Education staff at PDE have an "informal" 
fcelationship with^special educatiou .staf f • For example, there are 
occasional telephone calls, periodic meetings, and joint task forces are 
established with representatives from both areas where work assignments 
^-require it. An example of the latter is. the special education audit form, 
which has a vocational part. Vocational Education staff have also served 
om. audit teams. Several initiatives were started a few years ago but 
dissipated as a result of staff turnover. PDE staff from the Bureau of 
^Vjpca^tioual Education who were interviewed indicated that "they would ..like 
to see a more formal, joint effort with the Bureau of Special Education" In 
the ^.^pirit 'of interagency cooperation. At the LEA level, some complaints 
vere received about a vlack of " coordination between the Bureau of Vocational 
Education 'and other state offices. These were expressed particularly with 
regard to long fange planning, development of policies and position papers}, 
o{>er.at ions, planning, fiscal affairs, and administration. Some respondents 
indicated that the LEAs cpuld use special education dollars to do things 
that vocational education funds won't permit them to do because of funding 
restrictions. Thi3 includes identification, assessment, and pre-vo'cational 
training for handicapped students. 

'In the 1980-81 state plan, five recommendations pertinent to this 
topic were offered: (1) that formal interagency agreements between 
vocational education, special education, and vocational rehabilitatiou be 
^ established; (2) that special education staff review all vocational • 
education projects tQ insure no duplication of efforts; (3) that local 
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vocational education staff participate in lEP development; (4) that the RCU 
develop film and 'materials; (5) that teacher in-service activities be 
enhanced* While there is no indication that the first recommendation was 
ever implemented, some of the others have been undertaken to a limited 
extent. For example, lUs now need to sign off on vocational education 
projects for the handicapped in order to avoid duplication of efforts. 
Although there may have been more formal cooperation between agencies 
during 1979-80 and 1980--81, none is strongly in evidence at the present 



Project outcomes are probably among the most important areas of focus 
for the current study, but also among the least documented aspects of the 
program at all levels from the state down through the local. Not only is 
there a dearth of evidence %available with respect to project outcomes, but 



there is also a largely misunderstood notion of what constitutes outcome 
evaluation. In fact, the notion of evaluation itself is often confused • 
among program personnel with the concepts of assessment and individual 
diagnostic testing. One project, after being asked to provide additional 
information on outcome evaluation, supplied a 208 page final evaluation 
report. This evaluation report consisted of copies of diagnostic tests 
taken by participating students (some of them blank) together with 
individual instructional prescriptions or samples thereof. The entire 208 
pages was devoid of any process or outcome evaluative information. Strong 
recommendations concerning the need for state leadership in the area of 
outcome evaluation are presented in a later section. In the present 
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section, however, questions with regard to project outcomes are addressed 
in as comprehensive a manner as permitted by the scarcity of data available 
to be surveyed. 



Questio ns 1 and 2, Have the needs of the target population been met? Are 
needs met because of PDE funded projects or other factors? 



To address this question^ data from the mail survey and interviews 
were chiefly utilized. Using these data collection techniques, the ^ 
question was posed as to whether the projects v^th which the respondents 
were connected met the intended needs. Given the general nature of the 
question and the self-report aspect of it, it is not totally unexpected 
that the positive results shown in Table 11 were obtained. 

Examining this dichotomous self-report of success more closely, both 

the mail survey and interview data collection techniques provided 

independent estimates of program outcomes with respect to three positive 

types: generally positive outcomes, graduates successfully employed, and 

students mainstreamed into regular vocational education. Table 12 

indicates the analysis results for each of these three outcome types for 

both the mail survey and interviews. For each outcome type, the percent of 

the total number of programs sampled responding is indicated, and the mean 

estimate of program impact with respect to that outcome category is 

presented. Data on responses from the personal and telephone interviews 

were based upon the sufficiency of evidence for making a decision or an 

estimate with regard to each of the outcome categories. That is, the 

response rate represents the percentage of programs in the sample for which 

sufficient evidence was deemed present by the interviewer to warrant 

J 



ERLC 57 





Table 11 






Have the Projects Met the Intended Needs? 




Response 
^ ....... ^ 


, Mail Survey 


Interviews 


Yes 


100% 


100% 


No 


0% 


0% 
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Table 12 

» 

Types of Project Outcomes 



Outcome 



Ma ll Survey 
Response Rate Mean Impact 



. Interviews 



Response Rate Mean Impact 



1. Positive Outcomes 



74% 



81% 



84% 



21 Graduates Success- 
fully Employed 



44% 



49% 



44% 



57% 



3. Mains treamed into 
"Regular" Vocational 
Education 



65% 



64% 



23% 



64% 
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inclusion in the mean impact estimate. As might be expected, the response 

rates for the interviews were lower than those from the mail survey. This 

was probably due to the more stringent requirements for avail;able evidence 

which were applied by interviewers before including the data, as contrasted 

this with a lack of any requirements 'on respondents to the mail survey with 

respect to sufficiency of evidence* However > the impact estimates for mail 

survey and inttsrview data collection methods are remarkably similar* By 

replicating the mean impact estimates in this fashion through two 

independent data collecti'oh procedures, considerable credence and 

••.onfidence is placed in the obtained results* TFks leads to the 

■ \ 

gianeralizations that of all of the handicapped students participating in 
vocational education programs > about 80% of them have seen some type of 
positive outcome,, with approximately 50% gaining successful employment upon 
graduation and approximately 64% being' mainstreamed into regular vocational 
education programs* ^ 

. > 

Quest idns 3 and 4* V/hat is the eyidence of project i mpact? What programs 
Wo r k be st to r whom. 

While independent verification through separate surveys of the 
positive effects of vocational education for the handicapped represents 
something more than collective opinion, on the other hand, it reprfisents 
considerably less than what would be considered an adequate evaluation of 
program outcomes from a policy-making perspective* For this reason, data 
were examined on the kinds of evaluations which were conducted by funded 
programs in an effort to document any need for more formal evaluation 
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requirements, policy, or guidelines for rocal program^ from the state 
levels 

For purposes of this study, a vocational program for the himdloappcd 
was considered to have coiiducted a "formal evaluation" if information on 
process and outcomes was on hanfd, ready for use and decision-making, and 
such information: 

# had been collected on relevant project and state goals 

• ^ had been properly aggregated, analyzed, and interpreted 

• included comparison to no-project conditions 

# had been compiled in some written statement which included 
I results and conclusions* 

Once again, data from the mail survey and interviews were used to 

provide information on the e:ctent of fonn^l pro£;ram evaluation conducted 

within the vocational education funding program for the handicapped. All 

programs within the separate samples for the mail survey and interviews 

were categorized wit^^ respect to the following types of evaluation evidence 

present for their individual project; evidence of formal evaluation, 

formal evaluation reported, no formal evaluation, and no response. Table 

13 presents the results of analysis of this information* As can be seen 

from the table, only between 5% and 11% of the projects show any evidence 

of formal evaluation* Thi^ seem^ disturbingly low in a program which is"^^^ 

spending more than 4 million dollars per year in Supporting projects of 

this type. 

With respect to Table 13, it should be .noted that the substantial 
differences betv/een the mail survey and personal/telephone interview 
figures for the "forinal evaluation reported" and the "no formal evaluation" 
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Table 13 

Projects Conducting Formal Evaluation* 



Mail Survey Interviews 



■f ■ ■ ■■■ ~ 

Evidence of Formal Evaluation 11% 5% 

Formal Evaluation Reported ' 51%' .. io% 

No Formal Evaluation Reported 46% 90% 

No Response 3% . d^K^ ' 



*The second through fourth response, categories are mutually .exclusive and thus 
add to 100%, while the first category is a subset of the second category. 
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categary were probably attributable to the greater stringency with which 
interviewers questioned the project staff undergoing personal and telephone 
intervi'ews. On the raail survey, respondents were free to interpret loosely 
their responses as to what constituted a formal evaluation. Project staff 

4* 

were less I'ikelyj during a fonnal interview, to give the interviewer 
lAiformation that could be in any way challenged. This speaks well for the^ 
validity of the personal and telephone interview methods of data 
collection. 

In the absence of data on outcome evaluation available for projects 
funded under the vocational education for the handicapped program, certain 
indirect indicators of program impact may be examined. One of these is the 
extent to which local projects have produced a "ripple effect/' That is, 
to what extent has the project ^^een in demand in districts outside of the 
home dist/rict a^s indicated by requests for information, for presentations, 
or for technical* assistance in adopting or replicating the program. These 
may suggest that the positive effects the program has had, however 
documented, are sufficiently impressive to warrant the attention and the 
dedication of resources of other districts toward adopting or replicating 
thi^' program for their own use. Information on this effect is presented In 
T^ible 14. The results presented in this table are based^n an anal^/sis of 
data from personal and telephone interviev/s. Those interviewed were asked 
whether or not their projects^iytn^ced a ripple effect. They were also 
required to give evidence or further describe the kind of ripple effect. 
Based on their elaborated response to this item, analyses of the data were 
undertaken. Their responses were grouped into four categories: yes, no, 
maybe, and not applicable. The results indicated that for at least 37% of 
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Table 14 . 



Have the Projects Produced a Ripple Effect? 
Response ^ Percent Responding 



Yes , 37% 



^- No ^ • . 58% 

Maybe _ » 4% . 

Not Applicable . • 0% 



/ 
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the programs there was active interest from outside the home district in 
information and/or technical assistance necessity in adopting or 
replicating the program* 




Question 5 , Have projects developed l ocal c apacity to continue to meet 
needs of t arget group? 

This question is particulatly important in view of the alterntive 
capacity building vs* consumptive approaches. to funding discussed in 
relation to one of the earlier evaluation questions on PDE priority 
solutions (see components II, questions 1 & 2) • 

Based on data secured from the personal and telephone interviews, an 

analysis produced the tabulated -results shown in Table 15* This table \. 

reflects the percent^^e of projects indicating whether they could continue 

operations without state funding* Almost a quarter of the project.^ 

indicated that they could continue without any further state fur/ding* When 

♦ 

the percentage of projects which responded with a "maybe" is added, the 
indication is that as many as 40% of all of the programs currently funded 
under tha vocational education for the handicapped program could continue 
without any further state n nding* This result needs to be clarified in 
certain respects, of course. The programs were certainly not responding c>n 
the basis of continuation of their local projects at the full level of 
operation and service which they currently maintain* There is, therefore, 
no way of projecting the actual numbers of students which could be served, 
or conversely which could not be served, under circumstance-s in which no 
further funding would be available to projects currently operating. 
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Table 15 

V 

Could' the Projects Continue Without State Funding? 

Response . Percent Responding c 



Yes 24% 

No 51% 

V 

Maybe 19% 

Not Applicable 6% 



V 
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Nevertheless, from a capacity-building standpoint it is encouraging t\\ai 
the self-report data indicates evidence pf permanent effects of funding. 

' Policies 

■ • ^ 

The following study questions address various areas ^ in which future 

directions and policy recotmnendations can be mad,e^ re^l^rding the I^ureau of 

Vocational Education's vocational education program for the handicappj?d. 

As this section is^part of the analysis chapter, suggestions for future 

it 

directions and policy reconimendations will be limited to those directly 
represented in the data collected in connection with the stfudy. A 
synthesis of the data and information presented^ in this section, together 
.with RBS' own suggestions in -these areas, is presented in the later chapter 
dealing with, conclusions and recoaimendations. ^ ' . 

Questi ons I and 2> What issues or act ivities requir e fu r ther attention? 
What recommendations for future policies /pra c^t can be made? 

Based on telephone and personal inierviews as well as information 

gained through the mall survey, the following were indicated by respondents 

as issues or activities relating to vocational education of the handicapped 

tliat require further a.ttenpbtfft or recommendations for future . 

policies/practices* ^ . 



• The method of funding of vocational education programs for the 
handicapped. PDE ought to fund programs based on the 'quality of 
the program^ rather than the sheer number of students enrolled or 
the economic conditions pre^vaijling at the* local applicant level. 



The need to look at work directions other than manufacturing for 
the placement of the handicapped. 

;> 

The need to examine ways in which vocational programs for the 
handicapped can be kept current. 
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The need for a standard''^perational definition of aainstreaming. 
PDE needs to generate a standard definition of mainstreaming, as 
the 'present one is inadequate in several respects. ' ' • 

The need to move away from an emphasis upon p^roduction* data. 

The need for dissemination of 'information about .successful 
•projects. ' PDE needs to build funds for this into Jocal program 
bkidgets. * 

T^lje need for more stringent^ Evaluation of projects. PDE needs to 
bui^ funds for this into l\)cal program budgets. 

The need f^r closer monitoring* of local ^)roject ope^itions, PDE 
needs to institute a program nionitoring procedure which fosters 
more' aecountability . ■ 

The need to move. away from a definition of vocational education - 
for the handicapped which simply means "jobs." 

The need to provide opportunities &nd application guideline'is 
which go beyond the. traditional and are morfe creative -in,, 
programming. PDE'needs to broaden the application guidelines for 
funding. ' These application gui^clelines are currently too 
restrictive and do not allow for non-traditional or Innovative v 
■approaches to vocational programming .for the handicapped. 

The need for more follow-up evaluation and examination of 
enrployment results* PDE'needs to provide for more folk)w-up of 
outcomes of programs at the local level in order to mor^ closely 
examine employment results* ^ ^ 

The need to provide more opportunity in programming for 
incorporating pre-'employment t raining so handicapped 'students 
will be better prepared for specific skills training* * 

The ne'ed to provide racra vocational programming f or* haiidicapped 
girls. ' * . 

The need to provide more opportunity for remedial work, via 
computer assisted instruction. 

The need for more f leMbility .iil the application of funds. 

The^heed for bette/management information system information, 
PDfi needs to itedesign. its management information system relating 
to vocational education for the handicapped,. Current VEMIS are 
inadequate ^or use by local' programs in that they are generally 
believe4 to be of little use, somewhat inaccurate, and about two 
years out of date, ^ 
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« The need for better information with which to do long range 
planning. 

i» The need for vocational education to give more emphasis to 
inter-^figency agreements. 

^ The need for better definitions and standards regarding 

appropriate vocational education programs and activities for the 
handicapped. PDE should produce clear definitions and standards 
regarding what is appropriate for vocational programming for the 
handicapped leading to graduation. 

The need for an increased accountability among locally funded 
programs for vocational education of the handicapped. PDE needs 
to require final reports from locally 'funded programs which 
' include evaluations. PDE should provide for more accountability 
in the use of funds for each grant recipient. 

0 The need to 'p^o^ote more innovative programs for vocational 
education of the handicapped.- 

0 The need to provide more flexibility in expending funds. 

• The need to relax the certification requirements for vocational 
education personnel to realistic levels* 

9 The need to better address the social, needs -of special education 
students. 

p The need to share information among locally funded programs. PDE 
should provide a directory of programs that are fu^ed for ' 
vocational education of the handicapped, describing what ^ach 
local program is doing. 

« The need for better evaluation and needs assessment at the loca; 
level. 

# The need for less redundant and more streamlined application 
gu;Ldelines. 

« The need for better inservicing of vocational teachers on the 
■.ic'c^^ds of handicapped students. PDE shoulc^require regular 
vocational education faculty to receive inservice instruction on 
the teaching and management of handicapped students in vocational 
education. 

9 The minimum wage issue - if the minimum wage *is lowered there may 
not be enough incentive for employers to use the co-op education 
programs »^ especially in economically depressed areas. Even if 
the employers would hire more students, students would most 
likely quit school for full time work. 



( 



ERIC 



" 83 



All vocational education teachers need exposure to handicapped 
programs. 

The need to provide more competitiveness in the awarding of 
grants ♦ 

The need to provide extra funding to programs that have 
demonstrated they are exemplary* 

The need to better identify current local employment needs in the 
community and then to provide training appropriate to meeting 
those needs. 

The need to document the cost effectiveness of vocational 
prpgrams for the handicapped in order to justify increased 
funding. 

The need to better educate new co-^op employers and to provide 
more public relations materials on effective programs. 

The need for more intensive work on placements and attitude 
changes and job development among potential employers. 

:?fhe need for more communication and exchange of information on 
'problems among programs. PDE should institute regional meetings 
of s^^railar personnel to share ideas in vocational programming f«r 
the handicapped. PDE should provide for a means of sharing 
resources and teacher materials among local programs. 

The need for identification and dissemination of information on 
exemplary programs in vocational education for the handicapped. 
PDE should provide more technical assistance to locals in the 
area of programming versus applications for funds or fiscal 
monitoring. PDE should provide a means for more publicity for 
effective local programs. PDE should provide information on 
pieces of programs that are found to be effective and can be 
replicated elsewhere. 

The need for more programmatic technical assistance from state to 
local programs. PDE should provide more technical assistance to 
locals in the area of programming versus application guidelines 
for funds or fiscal monitoring. * 

The need for. revised timelines for notification of grant 
acceptance and receipt of funds in order that delays in start--ups 
are not produced by a late forwardi^ng of funds. 

Grant application materials should be sent to the project 
director in addition to the sponsoring institution in order to 
avoid the possibility of missing the grant because of getting 
information forwarded too late to <-pply. 
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« PDE should utilize the inter-agency agreement vehicle for 
f^Qcilitating programming. 

# PDE should re-*evaluate the use of entitlements. There needs to 
be some kind of merit system for funding and a means for creating 
a resurgency in the proposing of innovative programs. 

o The graduation requirements almost prevent sufficient time for 
voccH:ional education with all of the general education 
Xfiquirements needed. 

# The requirement of 160 minutes of instruction is a problem In 
operating a program for juniors and seniors only. 

0 . The need for better screening of the handicapped in order to 
maximize their chance for success. 

e The need for more money for computers-assisted instruction for the 
handicapped. PDE should provide more funding opportunities for 
equipment, especially computers, and mini^grants for use of 
computers-assisted Instruction with the handicapped. 

# The need for better communication between the areas of vocational 
education. and special education at the state level in order to 
avoid the frequent conflicting memoranda and messages coming to 
local programs independently from these areas. PDE should foster 
better communication between vocational education and special 
education areas within its own departm-int in order to provide a 
more unified approach to education of the handicapped and reduce 
such conflicts and contradictious in communications to local 
districts. 

» The need for a standard vocational education curriculum for 

various handicapping conditions, especially for mentally retarded 
handicapped students. 

o Special education teachers need to be certified in vocational 

education to offer vocational education courses, but vocational 
education instructors need no ^special education certification in 
order to teach special education students in a mainstreamed 
environment. This seems contradictory. PDE needs to lower or 
ease certification requirements for vocational education programs 
for the handicapped. 



Questions 3 and 4. What should be PDEVs role in " arrying out its functions 
in regar d to: l eaders hip, fundijig., dissemination of Information, technica l 
assistance/staf f developmen t. ^ monitoring/evaluation/quality assurance? How 
c an PDE im prove in carrying out its current role regarding thQse 
dlmenf^ions? 
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Many of the recommendations provided by respondents, as indicated in 
quCiStions 1 and 2 above, relate to the various elements in the present 
question. The. following paragraphs discuss each of the elements in light of 
recommendations received from the field through personal and telephone 
interviews, and through the mail survey^ 

With regard to PDE's role in terms of leadership, it was felt among 
local program staffs that PDE should take a more proactive xole in 
providing leadership for vocational education programs for the handicapped. 
PDE should be more than a gate keeper for the distributionvof federal 
fundr*. Rather, PDE should: establish standards to enhance the overall 
quality of vocational education programs for the handicapped in 
Pennsylvania; provide for the. identification, dissemination, and 
replication of exemplary program techniques; revise application guidelines 
in a way that would stimulate development of innovative programs; revise 
funding requirements to provide for more quality control, program 
evaluation, and greater local program accountability; and take the 
initiative in providing more programmatic technical assistance to local 
programs. 

V/ith respect to funding, local programs generally saw PDE's role as 
needing to provide a more equitable distribution of funds, and one that 
wouldjinsure innovative initiatives and quality in the programs funded. 

Substantial feedback was received from local programs with regard to 
the dissem>.aation of information* PDE was used to take the lead in 
identifying exemplary programs which are shown to be effective in 
vocational education for the handicapped. Once these programs have been 
'■ identified, PDE should take steps to insure that information with regard to 

ERIC 72 \ 



functions described in the preceding paragraphs. These sugge.stlons, while 
not comprehensive in their coverage, are presented below. 

With respect to leadership > the following suggestions were offered by 
respondents. PDE should develop a standard definition for mains treaming, 
and for the concept of vocational education especially as it applies to the 
handicapped. PDE should* allow jn ore em^ >g!sTs to be placed on pre^-empXoyment 
training in addition to training in specific job skills, PDE should also 
encourage th^, use of computer assisted instruction with the handicapped. 
In its leadership role, PDE should provide more comprehensive and more up 
to date management information system reports on vocational education for 
the handicappe<^. This would enable more effective long range planning to 
be undertaken. | In addition, PDE should develop standards regarding 
appropriate program activities for vocational education for ,the handicapped 
wMch would contribute toward graduation.^ PDE s|iould take staps to develop 
a formal vocational education curriculum for various handicapping 
conditions. 

With regard to fCnding, PDE should base its funding distribution 
policies more on the quality of programs being proposed rather than the 
local economic and unemployment conditions, or the number of students or 
student population within the dis;trict. It should build into local program 
grants fundsj for evaluation and dissemination. PDE should broaden its 
funding guidelines to allow non-traditional activities and programs to be 
proposed. The Departra.eivfc— PDE should allow pre--employment training 
activities in its funding guidelines as well as specific skill, training. 
Its funding guidelines should Include requirements for an annual report and 
an annual evaluation,, at the local level. More flex.^bility should be 
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allowed In the spending of funds on the local level. The grants 
procurement process should be competitive as opposed to formula-based. PDE 
should, revise its funding application timeline and seek to move up the 
deadlines for grants acceptance and facilitate the transfer of f.unds to the 
local programs in order that they may operate efficiently. Finally, PDE 
should send, application materials for n^w grants to the project director 
instead of (or in addition to) the funding agency so that the appropriate 
•person is notified in time to meet the deadlines. 

PDE can improve the dissemination of information by building funds , 
into eaqh project for dissemination. Final project reports and evaluation 
reports for each projecji^ should be received in a form that can be 
disseminated. PDE should promote conferencing and the* sharing of 
Information among projects. By providing a list of all projects and what 
they are funded for, PDE could promote more communication among programs 
and the exchange of infor^tion regard;ing problems. PDE should identify 
and disseminate information on exemplary programs. Finally, PDE should 
organize regional meetings concerning exemplary projects and practices. 

respect to technical assistance and staff development, PDE should 
provide' for more inservice instruction of regular vocational teac-hers on 
the needs and the instructional techniques to b|e used with handicapped 
etudents. The Department should pijpvijii more technical assi|tance in 
content and programmatic areas, and more PDE funding should be provided for 
staff development and inservice activities. 

With respect to monitoring, evaluation, and quality assurance, PDE 
should • rev-? se its monitoring system to. provide for greater quality 
assurance. Guidelines and funding for annual' evaluation of programs at the 
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local level should be provided. PDE should require an annual report which 
includes process and outcome evaluations for each project. Finally, PDE 
should change the method of distribution of funds In order to promote 
greater quality among projects • 



Question 5, What Criteria Should PDE Employ in FundlnR Future Programs for 
the Handicapped ? 

Feedback from interviewees regarding funding criteria for vocational 
education pr&grams for the handicapped was varied. Generally, most of the 
suggestions regarding .criteria for funding, as indicated in the previous 
three study questions, advocated a change in funding criteria from 
entitlement to come form >f funding based on th^ quality of the program 
being proposed ♦ Of course', these suggestions were coming from local 
districts who w^re receiving less funding, than they might have desired. No 

suggestions for changes in funding criteria were receive^ from districts 

\ . . ■ " 

benefiting in large measure from the present method for allocation of 

.funds. However, in a number of cases it is clear that suggested changes in 

funding crite? ^.a were based less on a concern for the amount, of funds being 

recel^ved currently by the district ^nd more on the need for improving the 

quality of programs and the accountability of 'local districts for funds 

being received* Further discussion with regard to these issues is provide^ 

in the conclusions and recommendations ^'chapter^f this report* 



IV. CONCLUSIONS AND FECOmNDATIONS :} 

I 

This chapter states the conclusior.3 and reconunendatious reached by 
^est^arch fpr Better Schools in carrying out the evaluation study and 
interpreting the results of the various analyses of data collected during 
the study. Both conclusions and " recommendations are presented in 
accordance with nhe major objectives of the evaluation. The conclusions 
represent inferences derived from a synthesis of results of analyses of the 
data collected',as part of the study as well as other background and 
ancillary information obtained during the conduct of the study. 
Recommendations are drawn on the basis of these conclusions. They 
represent rather concise statements ci the possible utilization of the 

X ■ 

results for making changes and improvements in policies, procedures, or 

resource allocations with respect to vocational education programs for th<^ • > ,. 

• ■ 

m^' ■ 

handicapped. . " • 



\ 

Problem s and Needs r- 



With respect to problems 'and needs in providing vocati/onal ^irfucation 
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services to the handicapped, the following conclusions are drawn. %M ' ■ 

It is not possible to "determine the extensiveness o. the targe: / j., _ 
population in Pennsylvania without consistency -of data collectiCT ^ 
methods in this area as well as a state-level definition of what ■ N' 
constitutes appropriate vocational education for the handicapped. 



Principal problems and needs center around equity in funding, 
program evaluation and accountability, inservlce instruction In 
special education for vocational \j:eachers» identification of • 
exemplary programs, curriculum standards, inter-p^"oject i^^„ 
communication, innovative programming, and greater inter-agency 
cooperation and coordination. These problems and aaeds have been 
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identification of exemplary programs and for providing the necessary 

publicity and dissemination to assist other local districts in learning 

about such programs is of paramount importance if the most is to be made of 

the investment in f cderal/sLate funds at the local level* Encouraging the 

adoption or replication of programs already proven effective by other 

districts with similar needs is cost-^ef fective* Similarly, inter-project 

communication about effective programs, practices, and problems needs to be 

enhanced. Shared problem solving among local districts on a state or. 

regional basis can also serve to maximize utilization of federal/state 

funds for local programs.' Such a sharing of effective programming^^ 

practices,., and ideas can also help to stimulate more innovative approaches 

to vocational education for the handicapped ap.d avoid repetition of less 

effective practices simply because of a lack of knowledge of anything 

better to use. The need for greater state leadership in the area of >^ 

curriculum and standards is also evident. There needs to be better 

Interagency cooperation and efforts toward establishing standard curriculum 

requirements for vocational education of the handicapped. 

Given the above conclusions related to the area of .problems and needs » 

the following recommendations are offered. 

1 . PDE should deve] "-p a s tatement of philosophy regardin g the 

appt opr.xateness r f vocational education for the handicapped and 
then p r es crib e stand a rd meth o ds for determining the handicapped 
stud ents for whom It would be appropriate . 

^ • PpS shoul^\ cortduct an annual field-based assessment of problems 
ilG^LilSS^ -n vocational education for the handicapped » and 
monitor pju* gr ess In addressing them * 
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Priorities 

V/lth respect to priorities and PDE*s approach to addressing the 

'i 

problems and needs of the handicapped In the area of vocational education, 
the following conclusions are drawn. * 

4 

# The existing PDE approach, and one It finds presumably to be 
desirable in attempting to accomplish its goals, is predominantly 
fiscal rather than programmatic* It also tends to be more 
consumptive than capacity-building in nature • 

0 State priorities are determined principally in a reactive rather 
than a .proactive way» . They are based, with few changes^ upon 
announced federal policies. State priorities are established in 
the'^'s'tate plan and, unless influenced by the state advisory 
/ council for vocational education or the advocacy groups for 

^ special education, remain largely the same from one year to the 

next* 

# State guidelines for funding of vocational education programs for 
the handicapped appear to be fairly clear and generally meet ;the 
information needs of local education agencies. State funding is 
distributed through an allocation method rather than a 
compe^titive procesTs. ? 

The PDE approach to vocational education for the handicapped appears 

to consist mainly of a procedure for distributing federal funds to local 
#» * 

projects. It does not repre^^ent a 'holistic- approach to improving local 

\ 

services which would advance the quality of vocational education services 
for the handicapped on a' state--wide basis. One characteristic of this lack 
of a holistic approach appears to be a focus by local projects upon more 
operational concerns rather than those that would contribute to and build 
local capacities to provide such services. One apparent result of this 
would be the fostering, through the distribution of federal funds, of a 
network of local services which would be continually dependent upon the 
receipt of such funds. An alternative, and seemingly more desirable, 
approa,ch would emphasize the building of local capacity to provide services 
even if support from the' state/federal government were diminished. 
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PDE priorities are determined simply in response to federal 
priorities. This procedure reflects a general lack of comprehensive 
planning and leadership on the part of the state. 

In most cases, state guidelines for funding of vocational education 
for the. handicapped programs appear to meet the needs of the local 
education agencies. The ^use of an allocation method or entitlements for 
distributing the funds has met with considerable criticism. There is 
sentiment among local districts that a more competitive funding process be 
introduced. 

Considering the above conclusions and their discussion, the following 
recommendations are made with respect to PDE priorities. 

3. In its funding. approach, PDE. should be m ore progra mmatic than 
fiscal and should place emphasis upon capacity-building ove r 
consumption . 

A. PDE should take a more proactive role in establishing priori'.ids 
for vocational education services for the handicapped a nd should 
go beyond a mere rest.atement of federal prioritie s in developing 
its own long-range goals and strategies. 

5. PDE should modify its funding guidelines to reflect more close ly 
its long-range goals and strategies for vocational education for 
the han d icapped . 

Activities and Proj^rams 

/ 

The following conclusions are drawn with respect to activities and • 
programs which are funded by PDE as' part of local projects for providing 
vocational education services to the handicapped. 

• The overwhelming majority (over 80 percent) of the programs are 
funded by the state at the secondary level through entitlements . 
and are continued year after year. The primary goals of such 
programs are; acquisition of vocational skills/competencies; 
mainstreaming into regular vocational classes; development of 
pre-employment skills; and placement in full/ part-time jobs. The 
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most frequent activitie^p of such programs are: specific job 
skill training; individualized programming; general vocational 
skills training; and pre-*employraent services. Little exists in 
the way of formal communications or relationships among funded 
projects. 

• About half of the activities within funded projects represent >^ 
what might be considereif "best practice*' in the field of 
vocational education services for the handicapped. 

> 

• In terms of funding distribution, the vast majority of funds 
..^ (over 80 percent) goes to support secondary programs, and 

' continuing rather than new programs ♦ In terras of services to the 
individual handicapping categories, these funds tend to serve the 
* the mentally retarded (59%); learni^ig disabled (25%); the 

emotionally disturbed' (10%); and the physically handicapped (6%). 

t Little in the way of interagency cooperation presently exists at 
the state level with regard to funding "or the administration of 
vocational< education projects for the handicapped. 

It is clear that the vocational education program for the handicapped 
is primarily a program for the secondary level, and its funds are 
distributed through entitlements* Projects funded at this level and in 
this manner tend to continue from year to year with little or no change. 

The fact that approximately hal^ of the activities used within funded 
projects represent what might be considered "be'st practice" is not 
surprising, considering that the emphasis at the state level is not upon 
providing -programmatic technical assistance to local projects, and that 
there is little interagency communication and coordination between the 
Bureau of Vocational Education and the Bureau of Special Education at the 
state level. There is considerable room for improvement in this area, but 
greater leadership at the state level needs to be exerted in order for this 
to occur.* 

The distribution of funds across local educational agencieo for 
providing vocational education services to the handicapped parallels the 
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manner in which progragi^ are distributed, that is, with most of the money 
and programs going to the secondary level and for continuation efforts. 

The general l^ck of interagency cooperation existing, at the state 
"level with regard to the funding and administration of vocational education 
projects tt)r the handicapped appears to be consistent with th6 reactive 
approach of the Bureau of Vocational Education to meeting the needs of the 
handicapped, and tha^redominantly fiscal approach used in distributing 
federal/state funds for such programs. 

In view of these conclusions, and their discussion relating to 
activities and programs, as presented above, the following recommendations 
are offered. ^ , ^ . • 

6. PDE should delineate "best practice", standards for vocational 
education services to the handica p ped and should require grant 
applicants to incorporate such standards into their local program 
applications . ■ ' ' ' 

"^he Bureau of Vocational Education should initiate communication 
and coordi nation with otHer bureaus within PDETeVg. , i^Pecial 
Educution) as well as othfei^ 'departments in x>rder to improve the 
quality of vocati onal education prog rams for the handicapped7 
encourage "best practice," and provide for the optimal use of 
vocational education services within the broader context of all 
services available for addressing the. educational, training, and ■ 
rehabilitative needs of the handicapped . 

O utcomes 

In terms of outcomes for vocational education projects for the 
handicapped, the present study leads to the following conclusions. 

• The re^eas_jo£^the target population are generally being met, with 
the majority b.f program participants achieving po'sitive outcomes, 
and about 50% of, those who graduate securing employment. 

• ^ It is n6t possible ""tp determine what programs work best for whom 

under what conditions''- because of the general lack of evaluative 
evidence concerning project impact. 
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« Less than half (about 40%) of the projects funded to pro\^ide 
vocational education to the handicapped indicated that they 
either definitely or possibJy could continue • their programs 
without any further state funding. 
• 

To th^ extent the target population can^be adequately identified^ a \ 
problem alluded to earlier, the needs of this' target population appeared to 
be met to some degree by existing programs for vocational education fot th^ 
'handicapped. This is based upon self-reports from local pjroject staff • 
either through mail questionnaire responses. or through personal or 
telephone interviews. Handicapped students appear to be achieving positive 
outcomes and receiving job placements upon graduation from programs. The 
extent to which federal/state funds contribute to the success of these 
programs, as opposed to local or matching funds, cannot be determined 
because most programs operate as complex interdependent efforts for which 
federal/state funding represent nearly inseparable cortribut ions. It is 
therefore impossible in practice to attribute effects of programs based on 
separate aljd from state/federal as opposed to local education agencies. 

Evii^ence of project Imls^ct for individual projects and for types of 
projects ^e largely based on self-^reported information, as mentioned 
earlier. It is rare that a project would have a formal evaluation- in place 
or woul^ have collected, analyzed, and prepared evaluation data in a foriu. 



.that could be utilized for inferring- project impact in an ^objective 
fashion.V F^wej^^^^than 10% of all projects under study were found to have any 




to determine what programs work best for whom under circumstances in which 

I 

there was virtually no objective evaluation data upon which to make such 



decisions* / 




* The fact that study results indicate about 40% of the projex:ts either 

^* ♦ 
definitely or, possible could continue their program services without any 

further state funding is difficult to interpret without any official state 

goals or strategies addressing capacity building or self subsistence* 

The ^bove conclusions and discussion concerning project outcomes has 

led to the following recoimnendations. 

8 . PDE shoul d spe cify principal outcomes for vocational education 
programs for the handicapped and should require all proje cts 't 
Lx epprLt,. the-jextent_of--achievemen^t^ ^ith 



substantiating documentation or evidence » 



9. PDE should require formal evaluations of all jro ^je cts. Projects 
would be required to specify formal evaluation designs' in thei r 
grant appjLica t ions» and continuation projects woUld be required 
to incorporate results of the yearns evaluation into a 

design or redesign of their' program plan for .the succeeding year. 
This would make possibl e analyses of the attainment of objectives 
of individual programs as well as types of programs * 

1 0 . PDE should determine the level of emphasis needed' to be placed ^ 
upon capacity building at the local level and should i ncorporate 
suitable requirements into the funding guidelines * 



Policies 

With re'spect to future directions and policy recommendations 

concerning PDE^s administration U3if state and federal funding for vocational 

education for the handicapped, the* following conclusions are drawn, 

♦ 

0 Issues and activities requiring further attention seem to be 
strongest in the following areas: programmatic technical 
assistance; allocation of federal and^ state funds; exemplary 
program identification and dissemination; program evaluation and 
quality assurance; innovative programming; inte,' -program 
communication; interagency cooperation; and curriculum and 
standards. , ^ 

• Respondents to surveys and interviews felt -that: PDE should take 
a greater leadership role in vocational education for the 
handicapped; funding criteria and method of aJLlocation should be 
changed; greater emphasis should be placed on dissemination; more 
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programmatic technical assistance and staff development should be 
provided; and greater monitoring, evaluation,* and quality ' 
assurance should be provided. 



Respondents to surveys and inter>4iews 
employ more quality criteria and more 
future programs for the handicapped. \ 



indicated -that PDE should" 
competitiveness. in funding 



A multitude of issues and activities requiring further attention were 
indicated by respondents to both surveys and inter^views. . A more specific 
listing , of these was presented fn the analysis , chapter of .this report. The 
first conclusion in this section lists those which could be considered 
major issues- or activities requiring further attention. 

A large variety of responses to survey and interview questions 
involving PDE^s role in carrying out their functions was received, 
specific ideas aud feedback with regard to leadership, funding, 
dissemination bf information, technical assistance/staff development, and 
monitoring/evaluation/quality assurance can be found in the analysis 

r. ■ . 

chapter of thi^ report. - ^ 

A large number of respondents to both sqrvey and interview questions 

indicated that the method^of allocation used ^by PDE to distribute " - 

state/federal funds needs to be changed. Specific responses, presented in 

the analysis chapter of this report, ranged from suggestions that the 

process become entirely competitive to vai^^ous othep less radical 

modifications and alternatives to the entitlement method. * 

Based on the conclusions and discussions above, related to future 

directions and policies, the following recommendations are offered. 

lU ■ PDK should exert a stronger leadership role in yocatioi^l 
educa|:ion services for the handicapped by promoting the 
development of model curric ula and inst ructional standards . 
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1 2 . PDE shoulu develop long-range goals and str ategie s based ^ 
upon an annual needs assessment conducted among local .programs , 

^ ^ • PDK should enhance quality assurance among l ocal projects by 
ins tltuHnfi more ytrlnp^ont repor ting rc^qulre m ents, Inc hut tng: 
annual final report^; process and outcome evaluations; cind, for 
continuation projects, program Im provement p lans for each 
succeeding year's applii:ktlon > 

1 A . Based upon good evaluation Information, PDE sjiould identify 

exemplar y programs and disseminate- Information about these 

programs to all local educational agencies > 
* * 

15.' PDE should somewhat reducer jts flsfcal and procedural monitoring 
and technical as s ista n ce, and should Institute a ^broader program 
for programmatic technlca-1 assistance to l ocal educa tion al 
agencies providing yocatlonal education services to the 
handicapped . 

16* PDE should encourage more staff development and In-s e rvlc p/ 
pre-servlce instruction for regul ar vocational edu cators In 
, providing for the handicapped^ especially under malnstre amed 
* conditions ^ 

17, To facilitate inteij -pro ject communication, PDE should compile, and 
distribute abstracts of all funded programs to all local 
educational agencies . 

^ ^ ♦ should re-examine its current methods and p-rocedures for 

funding programs, and should encourage more Innovation An 
I nstructional techniques and more competit iv eness in the 
a pplication proces s. ' 




V.' DISSEMINATION- 

^As of the writing 'of thi? evaluation report, some dissemination of the 

findings of thi& study has already taken place — a presentation of 

^^reliminary results was made at a session of the 1984 Pennsylvania 

* ' * '■ 

Vocational Education Conference. A paper proposal has also been submitted 

tqr the Vocational Education Special Interest Group of the American 

Educational Research Association for possible presentation at their -spring 

^1985 annual meetings ' ' • ^ / 

-The primary vehicle for dissemination of this final report, however, 

will be Pennsylvania *s Vocational! Educatiopi Information Network (VEIN)^ It 

is expected thaf'PDE wiil also submit the report for inclusion in the ERIC 

system^ In addition, an ExecuCive Summary, of ..the fiija^ report has been 

prepared for distribution* This summary , .with PDE approval, will be 

further disseminated to institutl^s,- a^clations,_5ind groups'", such as the 

' • . V 
following: . ^ » ^ 

♦ ^ Universities (e*g». Temple, Penn State, Plt^,*" all schools in the • 
^ state university/college ^ysUem) 

• Pennsylvania State Department of Eduj^atlon (various bureaus and' 
^ . unitiO ^ • ^ " * . 

• 'Intermediate Units 

# Urban. School Districts • * * . 

# ^National Center for Research in Vocatibnal Education (Ohio State 
University) ' 

e Center^ for Vocational Personnel Preparation (Indiana University 
of Pennsylvania) , - . . 

9 Pennsylvania State Advisory ^Commit tee for Vocational Education 

^ Vocational Administrators of Pennsylvania 
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0 PeniVsylvaBla Association of Vocational Education Special Needs 
Persqnnel 

V Pennsylvania Association of Voc^ational Teacher Educators . 

# Ameri'can Vocational Association 

# American Personnel., and Guidance Association 

' ' \ 

# Pennsylvania VocaMonal Association 

# Pennsylvania State Education Association 
« RISE* \ ^ . 

Also, RBS staff will cooperate with newsletters or publications of 
these groups' and will prepare press relaases, upon request. Finally, study 
bindings and recommendations \^ill be disseminated through existing RBS 
.dissemination channels^ such as the Research and Development ' Exchange* 



The major purpose of this widespread dissemination effort is to 



generate an awareness of the status of vocational programs fot *;he 
handicapped in Pennsylvania and the progress being made in this area. Such 
increased awareness will he^lp to mobii^ae groups fj^r acting on policy* 
recommendations and future PDE policies*. PDE^, however, will be the 
ultimat|/ beneficiary .of the. evaluation atudy, since results will provide a 
comprehensive data base for raiaking important policy decisions and the 
impetus to move forward on those decisio;;i^. 
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A. PDE Interview Guide 
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Handicapped Pro|2:ram Interview 



Person Intetviewed: 
Position Title: 



Interviewer; 
Date: ' 



1. What has been >RStnf role and responsibilities with respect to the 

Vocational Education Program for the Handicapped?- During what per^iod 
of .time have you had this' role and these responsibilities? 



2. How extensive is the target population for this program in- 
Pennsylvania? (I-l) , • -J 



3. What are th^ special- problems/needs of the target population?. H 



low 



would you concisely describe the background of the educational ' ^ind/or , 
training plight of^the target group? (I~2) 



4. Has. the need for such programs been reduced because of the success 
. these programs or from other external courses? 




5. Wliat is the existing PDE approach to solve problems within this 

program (i.e., stated goals, long and short term objectives, guide- 
lihes;>? Have the activities supported by PDE focused on priority 
solutions for the problems of the target populatlun? (II~1) 



6. 



What: are desirable policies/practices for PDE^ts. employ in order to 
accomplish goals for this program? (11-2) 



0 

. How were PDE priorities determined and validated? (II-3) 




What are the relationships (i.e., commonalities and unique^ features) 
between the various PDE obiectives for this program? Are there, 
planned complementary relationships between the objectives and 
activities of the .different subareas of handicapped funding? (TT.-4) 



■Do PDE guidelines .seem to meet the information needs of local 
education agencies with respect to this program? (II-5) 



What is the current PDE funding approach for this program? How 
appropriate is the fund distribution in relation to the target group? ' 
(II-6) 



what extent is there interagency cooperation (e.g., Bureau of 
Vocational Education, Bureau of Rehabilitation, Bureau of SpeciaV;^ 
Education, Research CoordiuatJ'ig Unit) in terms of funding and 
administering vocational education programs for the handicapped?' 
(IM~5) 
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12. Have projects developed the capar/ty to continue to meet the 
^ vocational needs of handicapped' students? What does the documented 
. ..evldenc(Mreveal with regard ^to improvement of the capacity of voca- 
r.onal education to nieot'^.the neednfof present and future target ponu- 
J/iMon« either with or v^lthout supplemental funding? (IV~5) 



13. What. issues or activities require further atte*ition in the future with 
regard to this program? (V-1) 



14. What recommendations for future policies/practices can be made with 
respect to this program? (V-.2) ™e,wicn 
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What should be- PDE's^ roiQ in carrying out their function, with regard 
to: leadership, funding, diiSseraination of '.information, technical 
asgistance/staff development, monitoring/evaluation/quality assurance 
(V-3) ... 



How can'PDE improve in carrying out its current role regarding the 
dimShsi^ons noted abo^^e in the previous question? .(V~4) 



What criteria should PDE employ in funding future programs for the 
handicapped? (V~5) 
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B. PDE Handicapped Project File Data Base 
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CODING SCALE 



PDE Handlcappe,d Fui\di^ Areas 



Subpart 2 

\_ Handicapped, Highfer Education 
2 Handicapped, Secondary 

Subpart 

5^ Guidance, Handicapped 

j6 Curriculum, Handicapped 

2 Staff Development, Handicapped 



/ 



fERJC 



mil 4 



PDE HANDICAPPED.. ACTIVITIES /SERVICES 



01 salaries and benefits 

02 equipment, specialized • 

03 equipment, installation 

04 equipment,. rental 

* 'v; 05 materials and supplies 

06 facilities,, rental 

07 contracted services 

08 transporation, travel 
09\^ other ■ 



v- 
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OBJ ECT I VES / COMPONENTS 




01 special/single skill training , 

02 buiidiVig trades training - maintenance 

03 auto repair - training 

04 'Tusi^ess ~ training - cler i^aU- retail sales ^ 
i ' ' ^ ■ 

al service— 'training . ^ 
ulture - horticulture - training . *. 
er skiii training " - 

*p8 diversified/multiple occupational skill training - general 
'industrial program 

09 entry level job skill trailing 

10 tooJ skills " h , ■ ■ ' . 

11 work coop/work experience 

12 academic program -^includes - .reading, math, language' ar^s 

13 home management instruction and institution ^ 'itlKludes food 
services, home cleaning - domestic .services / V 

14 individualized :^struction - support 

15 competency based instruction 

16 ""main^tr earning support ' . 

17 remedial component 

18 mainstreamed program 

19 modified program 

20 placement program 

^1 sheltered workshop program 

22 .workshops conferences 

41 TMR population or EMR population 

50 handicapped ~ voc. guidance - career counseling 
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VQC. FIELD - HANDICAPPED 



2V D.O. PROGRAM • 

19. COOP. PROGRAM . 

20 VOC PREP TRAINING 
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e. LEA Iift^rview S.ample 



V 
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Handicapped Project Interview. Sample 



• 


/A 


' " Title . 


Location 


Funding 


fype 


Site 
Visit' 


Telephone 


1. 

• 


76-1810 * 


Instructional Personnel f of the 


r 1 J- -. nr ' ^ . T II.. I „„. 

Berks Co. Avts, 
Berks Co. Avts 


21,615 


1 

Secondary 


X 




2. 


• 76-1811 


Vocational Special Education 


21,780 


S^.condary ^ 


X . 


*■ 


3. 


76-2002 . 


Vocational Proftxams for fhp 
Handicapped 


rnxxaaeipni^, ' 


7iOy602 


Secondary 
* * 'i • 1 


* 

A 


*• 


4. 


76-»2016 


Skill Devpl rvnropn f ^n 

Least Restrictive Environ. 


Bucks Co. lu f/2ii 


54 ,451 


Secondary 


X 




5. 


76-2019 


/Handicapped-Special Education 


E. Montc'o^ Avts 


• \ 
31,428 


Secondary 


' X 




6. 


76-2043 


Vocational F.H!lr*^>^^ nn % Ty'l^^^-l^r^r> 

for Handicapped 


Central- Susquehanna 




Secondary 


X 




-7. 


.76-2059 


Vocational Education & Training 


Capieal Xrea'lU #15 


■ 123,721 


Secondary 


X 




8.- 


76-2075 


Vocational Skills Development 
Program for Handicapped 


Chester Upland 


33,070' 


Secondary 


X 




9. 


76-2086 


•.^Vocational Education/Single Skills 


K^orristown Area 


24,367 


Secondary 


X 


- 


10/ 


76-1801 


Project Liaison, E^^R,TMR■, PH 


Pittsburgh 


23-3,463 


Secondary 


V 


X 


11. 


76-1804 


Food Services ' 


Delaware Valley Avts 


1,277 


S^condaty 




X 


12. 


76-1808 


Vocational Education for 
Handicapped Students 


Luzerne Co. lU /U8 


34,234 


Secondary 




X 

* 


n. 


76-1812 


Vocational Lab Assistant 


Reading-Muhlenberg 
Avts 


11,645 


Secondary 


* • 


X 

1 



14. 76 

15. 76- 

16. 76" 

17. 76- 

18. 76- 



-1816 
-2006 
•2011 

■201 
2023 



Title 



Option^ in Vocational Education 
for Handicapped 

Modification C Fo^d Service 
Program for Handlcfapped 



Locat ior. 



Allegheny iU //3 



Huntingdon" Area 



Funding 



Type 



19. 76 

> 

20. 76 

21. .76. 

22. -76- 

23. .76- 

24. 76- 

25. 76- 



-2028 
-2035 
-204; 

* 

-2047 
"2052 

■2057 

■2064 



26. 76-2069' 



Modified Skill Development 'Program 
for Handicapped - 

Vocational Education Training 
f:>r Handicapped 

Special Services for the 
Handicapped 

Aides for Handicapped 

Geu?.ral Industrial Program 

Special Shop. Program for 
EMR Students ' 

Handicapped 

Vocational Education for the 
Handicapped 

Avts/Special Education Liaison 
Teacher 

Modified Vocational Education 
Program for' Students with 
Special Needs 

Improving Basic" Skills Needed 
for Life Roles 
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Seneca Highlands , 
IU #9' ■ 

Carbon Co. Avts 

Bethlehem Area 

Lackawanna Co, Avts 
Greater Johnstown Avts 
Butler Area 

Upper Adams 
Erie City 



51 ,093 Secondary 



6,913 Secondary 



27,190 Secondary 



7 , 1 19. Secondary 



30*986 Secondary 



21,752 
37,044 



3,212 
36,934 



Merger Co* Avts 



Pennridge* 



Secondary 
Secondary 



8,200 Secondary 



Secondary 
Secondary 



Beaver Valley^ .IU' //27 19,700 Secondary 



24,706 Secondary 



6,204 



m 



Si^coudary 



itle 



Location 



Funding 



Type 



Site 
Visit 



Telephon: 



27. 76-207/. 

28. 76-2079 

2'9. 76-2g82 

30. 76-20)55 

31. '76-0814 
.32. 76-4007 

33. 76-1050 

34. 76r-1051 

35. 76-1065 

36. 76-V072 

37. ' 77-2003 

38. 77-2009 

39. 77-2011 



fRlC 



li.O. for Handicapped 



Instructor of Vocational Education 
cae jjandicapped-Tioga Co. 

D.O. York Co 

Support Services for^the 
Handicapped ^ ' 

i 

CooRfirative Diversified 
Occupation Program ' 

Multiple Occupation Education 

Diversified Occupations for 
Handicapped 

Vocational Skill Training 
for Handicapped 

Instructor of Vocational Ed. 
for Handicapped -Tioga' Co. 

Vocational for Trainables 

Vocational Studies for the 
Emotionally Handicapped - 



Postsecondary Food Services 
Training 

Cleaning Services Training ' 
for MR Adults 
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Conewaygo Valley 
Blast TU 

Lincoln lU /a 2 
Venango Co. Avts 

Greenville Area 



4 ,598 Secondary' 
18,612 Secondary 



47,736 
23,106 



C., Westmoreland 
Co. Avts 

^ Jefferson Co.- 
Dubois Avts 

Delaware Co. Avts 
t 

Blast lU //17 

* ^ 

Hazleton Avts 
. Montgomery CCC 

'( Elwya Inscitute 



Allegheny CCC, College 
Center-North 



Secondary 
Secondary 



5,370 ^ Secondary 

49,720 Secondary 

16,805 Secondary 

118,814 ' Secondary 

9,725 . Secondary 

12,318 Secondary 

63,366 Post Secondary 

21,486 Post Secondary 

49,832 . Post Secondary 



X 



X 
X 

X 
X 

vx 

X 
X 



X 
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D. LEA Personal Interview Form 



0 

1 
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HANDICAPPED SITE VISIT INTERVIEW 



Name 



Date 



School 



Interviewer 



Project 



A^ Introduction , / 

^ 1. Review information on project abstract. 



2. Describe gener^al purpose of project, objectives, and procedures. 



3, How many years has the project been running? Wha£ have been the 
major changes, over time? 



4.- Describe how project relates to vocational guidance (or handicapped) needs. 



,5. Describe what- improvement or advantage the project represents for 
the Pennsylvania vocational guidance (ot handicapped) system. 
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Project Characterist^ics (Describe each) 
1. Target Group - 



2. Activities 



;3. Staffing - 



4« Resources - 



5. Evaluation Procedures 



» 

6. How were the project activities selected? What evidence 
that they represent "best practice" in the field? 
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Process Evaluation 

1* Did evaluatiQn procedures provide formative information? 



2. Was the project implemented according to plan? Were there 
obstacles, changes? 



3. What factors helped the project proceed smoothly? 



4. Were the resources sufficient to accomplish project objectives? 



\ 



5. Did the project get sufficient i^upport from school and/or 
district administration? 
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Impact Evaluation 

1. Did evaluation procedures provide information on proj.ect impact? 



2« What was the impact of the project? Based on what evidence? 



3. What activities work best for particular . target population? 

* 

■f 1 

. -> . • \ . . 



4. Did the project meet intended needs? How were needs diminished, 
eliminated, or otherwise affected? 



5. Has the project produced "ripple effects on other schools"? 

/ ' ■ 



6, Can the project continue with diminished PDE resources? No PDE 
resources? 



\ 



116 



129 



E» Recommendations 

Do you have recommendations, for PDE procedures or activities? 
Do the guidelines me^t LEA needs? 



/ 



2^ Wha;: issues or activities require further attention? 



Interviewer Comments 
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E, LEA Interview Guide 



/ 



/ 
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PDE sitf; visit interview guide 

VOCATIONAL GUIDANCE & HANDICAPPED PROJECTS 



A. Introduction (suggested time ~ 10 minutes) 

• The general purposes for this group of questions are to provide a 
non-threatening l«ad-in to the interview, to . describe the overall 
context for the project, and to verify information on the pro3ect 
abstract. It should establish what the project intended to do. 



• 



Interviewer should preface the questions with a brief overview of the 
purposes of the study, RBS' role, and basic content of today s 
interview. Ask if there are any questions. Provide a copy of the 
RBS Annual Report. 

• Question 1: indicate that we have briefly reviewed sections of the 
projecF7~proposal and have compiled an abstract to describe it; have 
the respondent verify abstract data; however, don't get bogged down 
In details; in some instances it may be better to get the respondent 
talking about the overall' goals, objectives, etc. (Q.2) before 
mentioning data in the abstract. 

• Q uestion 2 ; self explanatory; make sure that the' answer is not too 
long; may want to tie/ in with Ql. 

• Question 3: self explanatory, note years before and after those ^ 
fcing studied (79-80, 80-81, 81-82); if ho longer operating ask wtiy; 
given time limits, focus only on major changes (e.g., at policy 
level) ; reinforce the idea that the interview is focusing on three 
specific years of operation. 

9 Question 4 ; What is the major problem that the project has 
- addressed? , ^ 

« Quest ion 5 ; The VEA specifies that PDE should help educatlonap 
agencies to address vocational guidance/handicapped needs thp^-ughout 
the state; how is this project part of %he "big picture ? What does 
the project "do" for the statewide system? Don't get into impact at 
this point. 

B. Project Characteristics (Suggested time « 15 minutes) 

« The general purpose for this group of questions is to accurately 
describe what the project did to accomplish its objectives. 

• Note changes over various project years. 

Ask if there are any descriptive documents that we could take back to 
RBS and/or review during the interview. 
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' ' ^ Qu<^-^^tlon 1 : whti did the project serve (numbers and descriptive 
categories)? \ . ' ' 

^ Questi on 2; briefly describe what was done;; although tliis is the 
^ bulk of this section, caa*t .spend too much time;, examples of issues 
by project type are: 

a. guidance, - components or features could include services provided, 
placement activities, counseling, establishment of resource 
center* etc.; describe typical student contact; determine if 
project involved ^'substitution" for prior servicea/activities 
(i.e», liiaintain guidance counselor). 

b. handicapped - services, type of instruction, curriculum, focus on 
pre-employment or' skill development, mainstream vs* sheltered; 
etc; describe typical da;^/week for a student, 

^ Qu estion 3 ; number involved, roles^ certification, (i.e/, 

special ed, voc. ed, jL;uidance% other), >otal FTE; note changes year' 
. to year; overview 4:ather than specifics* 

^ Question 4; what did they have to draw upon, e.g., facilities, 
equipment, technical support i etc.; supplementary resources (not 
included in PDE grant) , including additional funds. 

^ pue^tion 5 ; note that PDE and VEA have implied tlvat projects should 
^unilergo self-evaluation; have they done any evaluation? \i so, 
describe; obtain available reports; PDE requires iannual 
accountability repqrts and long-range planning efforts (LRPSI) for . 
certain projects; do they have anything? don*"t get into description 
of impact yet. • 

/ ^ Question 6 : how is what they've done exemplary in terms of what's 
regarded as "best practice" in vocational guidance/handicapped? 
e.g., for handicapped, must have ad^quatjg lEPsy should be 
Individualized, responsive to local job market, s^jiould have 
'^•N. coordination between special ed department and vocational ed 

department; may need to ask for"^ evidence to back-up statements' (e.g. 
lEPs, job market data, etc.); chi? is an Important question for 
handicapped. ^ 

C. ' Prdcess Evaluation (su ggested time ^ 10 minute^^^ 

« / Hie general purpose of this group of questions is to have the 

respondent provide subjective judgments about the success/failure of 
project activities/implementation. 

^ . Question 1 : note if responses to this section are based on post hoc ^ 
impressions or systematic;^ formative evaluations, i.e., what is the 
basis for their responses? ' 



4 
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• 9«ostlon 2; . what barriers, if any, prevented them from doing what 
they wanted to do? Are there any implications for other projects . 
based on their experienc^e? Don't get bogged down in documenting 
minute changes. 

• Question 3; reverse of Q. 2; what implementation factors have 
implications for other projects as facilitators of success? 

• Question 4; self explanatory; if no, document additional needs, how 
this potentially affected outcomes. 



Question 5 ; as appropriate; did organizational/agency factors affect 
implementation. 



D. Im pact Evaluation (suggested time ^ 15-20 minutes ) 

•The general purpose of this group of questions is to document the 
nature apd extent of program impact. 

♦ 

# This Is prpbably the most Important set of questions In the 
interview. 



• 



Different Impact areas ^re appropriate for different projects. 

Question 1; how did they gather what evidence of project impact? Is 
^the evidence har<| or soft? Obtain copies or review documented 
evidence • / <ii, ' - " » 

• Question 2 ; what are their claims of impact? Document the stlurce of 
'each claim; specifically, note impacts such as placement rates, 
completion ratesi achievement, raainstreaming (handicapped), skill 
development, affectiVe, etc.; get subjective impressions of where the 
project succeeded and where^ it failed. 

• Question 3 ; w'as there a relationship between certain activities and 
specific impacts? What activities seem to contribute most to 
parti^jular kindsVof imj^acts? ' , 

■ • Question A ; ' self-explanatoryi importanjt question given PDE 

"strategy"; if Wds not totally met, wljat are remaining needs? 

• Question 5; "ripple' effect" refers to spillover to other schools, 
staff, students, etc.; e.g., has the project had any effects on 
processes or outcomes beyond what was specifically targetted in the 
.proposal? Is it being Institutionalized? 

/ 

Question 6 ; important question, but touchj^ Issue; ^ssume that • 
diminished level of PDE <fesources for vocational- education, in ' 
general, is a. given; what would be effects of diminished /no PDE • 
resources upon the project? Aside from support of program operations 
has the project added to agency "capacity" to deliver programs o-r 
services? If so, how (e.g., improved staff capability, program 
development, improved facilities, equipment, etc.). 
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E. , Recomwendatlons (suggested time " 5-10 minutes ) 

e The purpose of these questions is to give respondents an opportunity 
to make suggestions regarding future PDE vocational education 
strategies. * " 

* Q uestion L ; the flsfet part of this question looks at recoi^hiendations 
for PDE roles, responsibilities, and/or policies (e.g., regardliig . 
funding, leadership, service delivery, quality assurance, etc.); the 
second part foouses specifically on the PDE application guidelines 
(and process); are they adequate and how could they be improved? 

' (Question '..2 : open-ended question for any other concerns. 
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F, LEA Telephone Irteyview Form 




Name: 



HANDICAPPED TELEPHONE INTERVIEW 
Date: 



Interviewer: 



^School: 
Proi ect.: 

• 1. Describe 'the .general purpose ^nd objectives of the project, 



2. ,How many years has the project, been "running? (Provide historical 
context.) _ ' . 



3. Describe how th^ project relates to vocational guidance /handicapped 
needs and how it adds to Pennsylvania's vocational education system, 
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Briefly describe project characteristics in terras of: 
a. Target 'Group ~ 



b. Activities (including how activities were selected) 



c. Staffing - 
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Did*you conduct. any formal evaluation activities? U so» dcsct tbo 
process and data sources (e*g,, hard vs* soft)* 



Was the project Implemented according to p4an? Were there any 
changes/obstacles/barriers (e.g,, level of resources, administrative 
support, etc-,)? 



What factors helped the project, proceed smoothly (i.e., facilitating 
factors witjh implications for other projects)? 
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8. What is the impact of the project (e.g., related to objectives)? 
based, on what evidence? ^^y^^f 



\ 



9. Did the project meet intended needs? Were overall needs of. tareet 
population diminished, eliminated, or otherwise affected? 



schooJs)?'^^''' ""'^P^^" '"-^^ on other 



11. Can the project continue with diminisl ed P0E resources? No P0E 
resources? What are the implications? 
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12. Dp you have recoraaendations for PDE procedures or activities? Do the 
application guidelines meet LEA needs? 



V 



13. vmat issues or activities require^ further attention? 



PENNSYLVANIA DEPARTMENT OF EDUCATION 
HANDICAPPED PROJECT EVALUATION 



Date: 

Educational Agency: ' 



Program Title: " 

Contract //: 

* -■ ■ a--ijij.i f--> .. I,. - , I - ,^ ■■ » . urTLf-i« u. i i iT>-ir ui ' 

. Of 

The purpose of this questionnaire is to provide information to PDE regarding the 
activities and impacts of vocational education programs for the handicapped. 
The form should be completed by the person most knowledgeable about your 
agency's program for the 1981-1982 school year . Please answer the questions as' 
best you can and return the completed form to Tom Blester, Research for Better 
schools. Inc., 444 Noi^th Third Street, Philadelphia, Pennsylvania 19123. (Return 
envelope enclosed.) 



^ 1. Wliat were the primary goals of your project? (Check all those that apply.) 

placement in full /4)art~ time job 

, acquisition of vocational akllls/compeitencies . , 

development of pre-employment skills 

staff development 

; mainstream into "regular" vocational classes 

other: 

other: 

^ I w ■« * " ' - - - - -- . J J» 



Briefly describe the primary purpose of the project 

L ^ ^ ^ ^^ ^.. . ^ . . _ ^ 11 , , ^ -..ji.-j ^ 

y . ---i 4-«^-F-v: I. _ini-..rT 

^ . ' ' » * 



2. Hov?^ many titxxdents actually participated in the project during the 1981-1982 
school year? 

^ # ^^l^ ^ 1? female ^ not- appropriate ^• 

3. Describe the types of h.^ndicaps for those students who participated in the 
program* indicating the approximate percentage for each type. 

Type of Handicap - Percent 



Name: 
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List the staff' assigned to the project by job title and percent assipment. 
Title v_ Percent Time Assigned to Proje ct 



What activities were conducted by the project? (Check all those that 
apply.) 

specific job skill training counseling 

^ pre-employment services remedial training 

^ placement services irdividuallzed program 

evaluation/assessment , ' self contained program 

competency-based instruction staff development 

mainstreamed program other support services 

general vocational skills other: 

work experience/coop. ' other: 



Briefly describe the primary project activities 



Did' you conduct any formal evaluation activities? 

yes no ^ 

It yes» please describe them briefly 



What was the overall impact of the project? 
If appropriate: 

% positive outcomes v 

' graduates successfully employed 

____ % mainstreamed Into "regulat" vocational education 

Or, please describe other posit^ impacts (e»g., related to specific 
goals /objectives of project), 



Indicate any coitmients or recommendations for PDE concerning vocational 
•education for the handicapped in Pennsylvania . 
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H» State Goals for Vocational Education 



132 



145 



STATE GOALS FOR VOCATIONAL EDUCATION 
IN PENNSYLVANIA. 

The State Board of Education recognizes .the increasinjj; com- 
plexitj^ and rising costs associated with providing educational services 
for all citizens of the Commonwealth. It is imperative that: (1) 
accountability be an intes;ral part of planning; (2) existing and pros- 
pective educational programs be monitored and evaluated; (3) outdated 
and unnecessary duplication of programs and services be eliminated; (4) 
educators iustify their educational expenditures to the public.; anc" (5) 
representatives of all educational levels participaLe. in determining KUe 
optimum delivery system for meeting educational needs of the public. 
Considering these points and, in an effort to provide a clear direction 
for vocational educators, the State Board of Education has adopted the 
following goals to guide vocational education for the next five years. 

I. Develop, expand or modify quality vocational education so 
that by 1982 every public secondary student will have had 
the opportunity to: (1) enter the labor force with a 
marketable vocational skill; (2) learn a useful voca- 
tional skill; or (3) acquire a basic vocational skill and 

' continue at the postsecondary level, 

II. Promote tiie expansion, range and diversity of adult and 
pos6eect)ndary occupational education opportunities to 
facilitate: (1) the entry/reentry of persons into the 
labor force and; (£') to provide upgrading or retraining 
for persons already employed/unemployed. 

III. Encourage alternative forms of vocational/occupational 
education to broaden the options available to students. 

IV. Intensify articulation efforts among programs at middle 

schools and junior high schools with secondary school and 
postsecondary vocational/occupational programs. 
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V. Support a greater understanding of educational/career 

options by expanding vocational guidance, counseling, and 
lob Placement services and providing occupational experiences 
for students. 

VI. Foster a concerted effortj to achieve equal educational 
ORPfJritunities by eliminatiing sex, racial/ethnic and 



li'n|*tiifetic bias, stereot;^ping, and discrimination. 

VII, Strengthen the development and implementation of compre- 
V hensively planned prnprams to meet the needs of and 

demands for competent professional vocational teachers, 
counselors, supervisors, and administrators. 

VIII. Develop and expand vocational education opportunities for 
the disadvantaged and handicapped. 
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